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Summary of Progress 


During the period June 1, 1988 - December 31, 1988, progress was made in the following 
areas: 

1) Performance Analysis of Bandwidth Efficient Trellis Codes on Channels with 

Phase Jitter 

In our January 1988 semi-annual report, we included some preliminary results on the effect 
of phase jitter on trellis coded phase modulation. We are now in the process of preparing a 
paper for submission to the IEEE Transactions on Communications summarizing our results 
on phase jitter. A preliminary version of this paper is included as Appendix A of this report 
[ 11 - 

Figure 9 in Appendix A contains a good summary of our results for coded 8-PSK and 
16-QAM modulation. When channel noise is the only cause of the phase jitter, i.e., the case 
of practical coherence, the SNR degradation due to phase jitter is less than with uncoded 
modulation. In other words, coding makes the system less sensitive to phase jitter. For 
example, with a phase noise standard deviation of 5°, coded 8-PSK suffers a degradation of 
ss 0.5 dB, whereas uncoded 4-PSK suffers a degradation of « 1 dB. On the other hand, when 
oscillator noise is also a factor, the performance degradation can be much worse, as illustrated 
in Figure 11 of Appendix A. 

Our work on the performance analysis of bandwidth efficient trellis codes on channels with 
phase jitter was performed by Dr. Christian Schlegel, a former Ph.D. student supported by 
the grant, who graduated in December 1988. Dr. Schlegel is now employed by ABB Corporate 
Research, Baden, Switzerland. 

2) Performance Analysis of Bandwidth Efficient Trellis Codes on Channels 

Disturbed by Jamming and Impulse Noise 

In our June 1988 annual report, we included some results on the performance analysis of 
bandwidth efficient trellis codes on fading channels. We have now extended these results to 
include channels disturbed by jamming and impulse noise. A paper containing these results 
was presented at the “Trois Journees sur le Codage” conference held in Toulon, France in 
November 1988. A copy of this paper is included as Appendix B of this report [2]. 

In this paper we analyze the performance of trellis coded modulation on channels with jam- 
ming and impulse noise. We assume a partial band jammer with an optimized duty cycle as 
a worst case impulse noise interferer. It is shown that the effective length, or minimum Ham- 
ming distance, of the code is the most important code parameter in determining performance. 
This is the same design parameter that resulted in the case of fading channels. However, 
the secondary design parameter differs in the two cases. Whereas for fading channels the 
minimum product distance was the secondary design parameter, for impulse noise channels 
the minimum free Euclidean distance fills this role. Some new trellis codes are constructed 


for impulse noise channels using these new design criteria. In Figures 5-7 of Appendix B, it 
is shown that they perform better than codes designed for the AWGN channel. 

Our work on the performance analysis of bandwidth efficient trellis codes on channels 
disturbed by jamming and impulse noise was performed by Dr. Christian Schlegel, a former 
Ph.D. student supported by the grant who graduated in December 1988. Dr. Schlegel is now 
employed by ABB Corporate Research, Baden, Switzerland. 

3) A New Construction Algorithm for Trellis Codes 

In our January 1988 semi-annual report, we included preliminary results on some hueristic 
construction algorithms for bandwidth efficient trellis codes with large constraint lengths. One 
of these algorithms has been developed further and has resulted in the construction of some 
good, long convolutional and trellis codes. A paper containing these results was presented at 
the “Trois Journees sur le Codage” conference held in Toulon, France in November 1988. A 
copy of this paper is included as Appendix C of this report [3]. 

The algorithm can be used to construct good convolutional or trellis codes with any con- 
straint length up to about 30, any signal constellation, and any code rate. No exhaustive 
search is required, so the complexity of the algorithm depends only on the difficulty of com- 
puting the free distance as the constraint length increases. As shown in Tables 1 and 2 of 
Appendix C, the codes constructed are either optimum or nearly optimum in free distance. 
The construction of long constraint length convolutional and trellis codes is important for ap- 
plications requiring extremely low decoded bit error rates, such as NASA deep space missions. 
It is anticipated that these long constraint length codes would be decoded using sequential 
decoding. The application of sequential decoding to bandwidth efficient trellis codes will be 
the subject of future research under this grant. 

Our work on this construction algorithm for trellis codes was performed by Dr. Marc 
Rouanne, a former Ph.D. student supported by the grant who graduated in May 1988. Dr. 
Rouanne is now employed by Matra Communication, Paris, France. 

4) Construction of Good Distance Profile Trellis Codes for Sequential Decoding 

Another approach to constructing good trellis codes for use with sequential decoding is to 
optimize the distance profile rather than the free distance. Chevillat and Costello [4] showed 
that this approach worked well for binary convolutional codes. We have taken a similar 
approach to the construction of bandwidth efficient trellis codes for sequential decoding. A 
paper containing our results was presented at the Allerton Conference on Communication, 
Control, and Computing held in Monticello, Illinois in September 1988. A more complete 
version of the same paper is in the process of being prepared for submission to the IEEE 
Transactions on Communications. A preliminary version of this paper is included as Appendix 
D of this report [5]. 

Using a nested step-by-step code search algorithm, we constructed 2-, 4-, and 8-dimensional 
trellis codes using rate k/k + 1 systematic binary convolutional codes with 8-PSK and 16-QAM 


signal sets. The Fano sequential decoding algorithm was simulated to study the effects of the 
distance profile on the computational load of a sequential decoder. An examination of Figures 
3, 7, and 8 in Appendix D illustrates that codes with a better distance profile can be decoded 
much faster using a sequential decoder. These results are important when selecting good codes 
for use with sequential decoding. 

Our work on the construction of good distance profile codes for use with sequential decoding 
was performed by Mr. Sankar Malladi, a former M.S. student supported by the grant, who 
graduated in December 1988. 

5) Comparison of Random Coding Bounds of Trellis Coded Modulation Schemes 

In our January 1988 semi-annual report, we summarized our work on minimum free Eu- 
clidean distance lower bounds for trellis coded modulation. Since then we have obtained a 
related upper bound on the event error probability of trellis coded modulation. A paper 
containing this new upper bound along with a comparison to the lower bounds on free dis- 
tance was presented at the European Signal Processing Conference held in Grenoble, France 
in September 1988. A copy of this paper is included as Appendix E of this report [6]. 

This new bound allows us to optimize a signal constellation for use with trellis coding by 
varying the signal points in the constellation until the best bound on event error probability 
is achieved. Figure 4, 5, and 6 in Appendix E show several examples of optimized signal 
constellations. These results are useful when one is attempting to find the signal constellation 
which will give the best performance with trellis coding. 

Our work on finding the best signal constellations to use with trellis coding was performed 
by Dr. Christian Schlegel, a former Ph.D. student supported by the grant, who graduated 
in December 1988. Dr. Schlegel is now employed by ABB Corporate Research, Baden, 
Switzerland. 
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abstract 

Bandwidth efficient data transmission using trellis coded modulation (TCM) relies on co- 
herent reception. Non-ideal carrier synchronization causes performance degradation. In 
this paper we apply the Chernoff bounding technique to obtain performance bounds for 
bandwidth efficient trellis codes on channels with phase jitter. It is shown that under the 
assumption of independent phase errors the performance degradation with respect to ideal 
synchronization is small. The independence of successive phase errors can be achieved in 
two ways: signal interleaving or the use of a decision-feedback predictor to whiten the phase 
errors. In the case of practical coherence, where channel noise is the only cause of phase 
jitter, a generalized distance is defined and generalized distance spectra are presented. The 
event error probability can be calculated for quasi-regular codes with an efficient algorithm 
. It is shown that this method can also be used to evaluate the influence of different causes 
of phase jitter. This is illustrated for the case of oscillator phase noise. 


L This work was supported by NASA Grant NAG 5-557. 


1 Introduction 


In a coherent trellis coded digital communication system, two essential functions must 
be performed by the receiver before the received two-dimensional signal vectors, denoted 
by t/ r , can be passed on to the decoder. First, the carrier wave must be synchronized, 
i.e., the locally generated reference waveforms must be accurately phase locked to those 
produced at the transmitter. This is called carrier recovery. Second, symbol timing must 
be recovered. This is referred to as timing recovery. The timing recovery circuit extracts 
a sinusoid of frequency 1/T from the baseband signal, where T is the baseband signal 
duration. The sinusoid’s phase determines the sampling instant and is important to proper 
system operation. Uncertainty in the frequency of the timing clock can usually be kept very 
small and timing recovery is less critical in terms of performance sensitivity than carrier 
recovery [1]. 

The carrier recovery circuit produces the local carrier signal cosfuV + #(<)), where cj 0 
is the carrier frequency and 8(t) is the phase error process. This local carrier signal should 
be sufficiently accurate so that the phase error is small in both its static (mean) value and 
in its fluctuations, which are called phase jitter. There exist several methods to recover 
the carrier frequency [2]. One method consists of extracting the carrier frequency with a 
phase-locked, loop (PLL). Another, decision- directed recovery, operates as follows. Due to 
disturbances, the receiver generated signal points y_ r will not lie at the discrete points of the 
signal constellation, but rather in small disc like regions around those signal points. The 
angular deviation of the received signal vectors y_ r from the points of the signal constellation 
generates sequences of control signals which are low-pass-filtered and applied to a voltage 
controlled oscillator, which then generates the local carrier frequency and phase. 

Decision directed carrier recovery is usually used for TCM due to its better tracking 
performance [3]. At the sampling instants the phase error process 0(t) leads to phase 
offset angles 6 r (Figure 1). The 0 T are approximately Gaussian distributed and strongly 
correlated due to the low pass filtering in the carrier recovery circuit [2,4]. The strong 
correlation has led to the assumption of a constant phase offset over the length of an error 
event in TCM. Hagenauer and Sundberg [5] show that such a constant phase offset results 
in serious degradation in system performance. 

In this paper we assume independent phase offset angles 9 r and show that the perfor- 
mance degradation is then relatively small. The independence of successive phase error 
angles can be achieved by interleaving, which is used to randomize the sequence of chan- 
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nel signals. We show in Appendix A that a linear phase predictior may also be used to 
whiten the residual phase errors. This preserves the advantages of decision directed phase 
sznchronization which have to be given up with interleaving. We will henceforth assume 
that the phase errors are independent. 

The phase errors can also have a static component, but it can usually be made quite 
small by the recovery circuit. It is much harder to minimize the fluctuations of the carrier 
phases, which result not only from channel noise but also from other physical phenomena. 


Figure 1: Effects of the r-th phase offset angle 0 T on the received signal constellation. 

2 Performance Bounds 

In this section we present a general method for deriving error performance bounds for coded 
systems used on memoryless channels 2 . We will apply these results to TCM communication 
systems whose structure is shown in Figure 2. A TCM communication system consists of 
a trellis encoder, possibly a signal interleaver, the transmission channel, possibly a signal 
deinterleaver, and a trellis decoder. A rate R = k/n trellis code is generated by a binary 
convolutional encoder followed by a mapper. The convolutional encoder is a finite state 
automaton with 2" possible states, where v is the memory order of the encoder. At each 
time interval r, the encoder accepts k binary input bits . . . ,uj) and makes a 

transition from its state S T at time r to one of 2 k possible successor states S T +\. The 
n = n — (k — k) output bits of the convolutional encoder and the k — k uncoded information 

2 This method can be extended to also include finite state channels. 
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bits {u k , . . . , u k+l ) form one of 2 n binary n-tuples v r = «, v?" 1 , . . . , v}), which is translated 
by the mapper into one of A = 2 n channel signals from a signal set A = • • • ,£u}- 



Figure 2: Trellis coded communication system. 

The uncoded information bits do not affect the state of the convolutional encoder and 
cause 2 k ~ k parallel transitions between the encoder states S r and S r + *. Since the coherent 
DSB-AM system transmits two dimensions in one analog waveform signal, it is sensible 
to design the trellis encoder for 2-dimensional signal sets. A rate R = k/n trellis code 
transmits k bits/channel signal, where the channel signal set contains A = 2" signals. If 
such a TCM communication system replaces an uncoded system that uses a signal set with 
A! = 2 k signals, the overall transmission rate is preserved, and we call such a TCM system 
bandwidth efficient. 

The particular transmission channel discussed in this paper is the AWGN-channel suf- 
fering from phase jitter. The linear phase offset predictor or the interleaver/ deinterleaver 
insure that the phase offset angles 0j in the received sequence are independent. They 
are decoded by a maximum likelihood sequence estimator (usually using the Viterbi algo- 
rithm). The Viterbi algorithm finds the signal sequence that most closely corresponds to 
the sequence of received signals. It achieves this by calculating the decoding metric m(x, y) 
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between x and y, where x = (&!, ...,£/) is a possible sequence of 2-dimensional transmitted 
signals and y = (^ l5 . . . is the received sequence. m(x, y) is some non-negative function 
of x given y, which is inversely related to the conditional probability that x was transmit- 
ted if y was received. The decoder will then choose the message sequence x for which this 
metric is minimized. It makes an error if it decodes a sequence x', given that the correct 
sequence, i.e., the transmitted sequence, was x. This will happen if m(x',y) < m(x,y). 

We assume at each step of the derivation that the receiver has no information about the 
phase error, since if it had such information, the received 2-dimensional signals y_ r could 
easily be rotated by the erroneous phase angle into the correct position. We first calculate 
the two code word error probability conditioned on the phase error 0, and then average 
over the density function of 0. Denote the two code word error probability when x is sent 
and x' is received, given the phase errors 0 = (#i, 0 2 , . . . , 0;) at each symbol interval, by 
P(x — > x' | 0). For the benefit of a more streamlined mathematical notation, we let x r be 
a complex number of which the real part is the in-phase component / of the signal and 
the imaginary part is the quadrature component Q. We further assume without loss of 
generality that the signals have unit average energy, i.e., Es — \Y1 P |%,| 2 = 1. In any case, 
we may always normalize the signal constellation and the channel noise by Es without 
changing any of the derived expressions. 

The two code word error probability, i.e., the probability that x' is erroneously decoded 
if x is sent is given by 

P(x->x') = £[P(x -» x' | 0)] 

$ 

P(x—*x') = BPr{m(x', y) — m(x,y) < 0 | 0}. (1) 

e 

We use the Chernoff bounding technique [7] to upper bound the above expression and 
obtain 


Pr{m(x / , y) - m(x, y) < 0 | 0} < E [exp(-A{m(x / , y) - m(x, y)}) | 0] , (2) 


where E denotes conditional expectation and A is a non-negative real valued parameter 

y |x 

over which we minimize the right hand side of (2) to obtain the tightest possible exponential 
bound, i.e., 


P(x-»x') < min EE [exp(— A{m(x',y) — m(x,y)})] 
A 9 y 
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(3) 


= min C(x, x', A), 

where C(x, x', A) is called the ChernofF bound between the signal sequences x and x'. 

Restricting attention to decoders using the additive maximum likelihood metric for the 
AWGN-channel, i.e., 

™( x >y) = ]Cl£ r ~^l 2 ( 4 ) 

r=l 

we may rewrite (3) as 

P(x — *■ x') < min C(x, x', A) 

= min EE exp(A(|x r | 2 - |4| 2 )) exp(-2ARe{?/ r (x r - a£)*}) 

x 6 y Lr=l 
l 

= min nCfe.i.A), (5) 

X r=l 

where C(z r ,3^!n A) is called the Chernoff factor of the signals x T and and * denotes 
complex conjugation. Due to the incorrectly received carrier phase, the received signal 
y r is given by y_ r = e i6r x_ T + n r , where & r denotes the usual 2-dimensional Gaussian noise 
vector expressed as a complex number, and i denotes the imaginary unit \/—L. After further 
manipulations we obtain 

C(x r ,xf, A) = E [exp(A(|x r | 2 - x^| 2 ) + A 2 JV 0 |x r -x^| 2 - 2A J Re{e u?r x r (x r -xj.)*})] . (6) 

In the next steps we will average (6) over the phase offset angles 9 r in the phase jitter 
case. The phase offset angles 0 T are statistically independent and identically distributed 
(i.i.d.). We then proceed to obtain for the Chernoff factor of the signals £ r and 

C(z„ il, A) = exp(A(| £r | 2 - Kl 2 ) + A 2 M,|& - lt| 2 )|:[ eX p(-2Afl e {e ls £r (l. - 4)’})] • (V 

Due to the stationarity and independence of the phase error process 9, we have dropped 
the dummy subscript r in the averaging over 9 above. 

As discussed in the introduction, the phase offset angles 9 r are assumed to be Gaussian 
distributed, i.e., 
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p(6) 


rv 


( 8 ) 


_J_ e -<> 2 / 2 °\ 

where a g is the phase offset angle variance. 

The analysis method presented here is not limited to phase noise induced by channel 
noise. Any combination of sources of synchronization errors can be included as long as the 
net interference can be modeled as a stochastic process which has Gaussian statistics. 

We now evaluate 

^[exp(-2A Re{e lS 2L r (x. r - ^)*})] = ^[exp(-2Af?e{(cos0 + isin0)£ r (£ r - £^)*})] . (9) 

For phase error angles 9 that are not too large, we may approximate the trigonometric 
functions by 

Q 2 

cos 9 « 1 — — (10) 

sin# « 9. (11) 

Using these expressions in the above evaluation of the expectation over 9 , we obtain 

£[«p(-2A7fe{e“ ir (*, - £)•})] = 

£ exp(-2Afle{(l - y + i0)x r (x, - a£)'}) 

= exp(-2Afie{i r (i r — ai)”})^ exp(-2Afle{(-^- + W)lr(Er - , 

( 12 ) 

and E[-\ is evaluated as 



Rewriting the signals x r and in polar form, i.e., 
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( 14 ) 


£r = U,|e'*' 

4 = 


we obtain 


i?e{i^. r (i r - xj.)*} = -/m{s. r (^ r - a£)*} 

= + /m{2. r ^.*} 

= |i r ||ij.|sin(^ r r»), (15) 

and analogously 

Re{£r(3Lr - J^)*} = \ 2 L rl 2 ~ l^rl |d£| COs(<£ rr «), ( 16 ) 

where <^ rr > = (f> r — <f>' r is the angle between the two signals £ r and x^. With these simplifi- 
cations we obtain for (13) 


fils 


i 

- exp 

yl - 2A<r|?7 r 


/ 2A 2 <Tg/c^ \ 

\1 -2\afo r ) ’ 


(17) 


where /c r = |x r || 3 ^| sin(^ rr <) and t] r = (|s. r | 2 — l^rll^rl cos(<£ rr »)). Using (12) and (17) in (7), 
we obtain for the Chernoff factors 


C(x,,£, A) = exp(— A(1 - XN 0 )\x r - 




\/l - 2A<r|77 r 


(18) 


The first exponential expression is the Chernoff factor for the AWGN-channel. The second 
term depends on the phase error 0 and reflects the degradation which a real system suffers 
from phase jitter with respect to an ideally synchronized system. 

The Chernoff factors are important because they not only streamline the expression for 
the two code word error probability but also apply to the transfer function bound for trellis 
codes introduced later and to the cutoff rate calculations. In particular, it can be shown 
[8] that R 0 , the channel cutoff-rate in bits/transmitted signal, is given by 


A A 


Ro(p) = -log 2 min J2p(n m )p(SL p )C(!L m ,tL p , A), 


m=l p= 1 


(19) 
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where p{(Lj) is the probability of choosing the signal € A. Note that R 0 is dependent 
on the particular metric m(]/ r ,£ r ) that is used by the decoder. If the decoder uses the 
maximum likelihood (ML)-metric for a memoryless channel, i.e., 

i 

m{ y, x) = - log(Pr(y|x)) = - log Pr(^J^ r ) 

r=l 

= S (- l0 S( Pr (^ r kr))) = (20) 

r= 1 r= 1 

(19) becomes the channel cutoff-rate for the optimum receiver, which is the usual definition 
of R 0 [7]. We will denote the value of A which maximizes the cutoff-rate (19) by A^,,. In 
this case, the Chernoff factors will be written as = C(a. m ,<lp,Xft 0 ). 

Evaluating (19) numerically, we observed that the minimizing value of A, A/j„, is very 
close to the minimizing value of A for the pure AWGN channel for <r|/iV 0 ss 0.25, i.e., 
Ai?o ~ 1 /2Ao. We therefore weaken the bound (17) and the Chernoff factors (18) minimally 
by setting A = 1/2 iVo ~ A/^ and the Chernoff factors become independent of A and are 
given by 

exp (21) 

J\ _ 

V 1 No 

The Chernoff factor in the general form of (21) is a function of the noise power spectral 
density Nq. The functional dependency of a$ has not yet been specified at this point. 


C(3L r ,sC) « exp (-^|£ r - 4I 2 ) 


3 Practical Coherence 


Following the terminology of [4] we call a system practically coherent if the phase noise of 
the carrier recovery loop contains only a channel noise induced component. Assuming that 
a first-order phase-locked loop is used to extract the carrier phase from the discrete carrier 
component ,we may use the results of [4, p. 35] for the phase jitter angle variance, i.e., 


2 _ NqBl 
Es ’ 


( 22 ) 


where Bl is the loop-noise bandwidth of the phase locked loop. The same expression for the 
phase jitter angle variance is obtained for suppressed-carrier tracking using an A-th order 
phase locked loop as discussed in [4, chapter 2] and [9]. 
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We now introduce the phase-noise-to-channel-noise ratio p = crjEs/N 0 . Note that for 
the two PLL examples cited above, p = Bl, the loop- noise bandwidth. For PLL’s, -j=^ is 
also called the loop signal-to-noise ratio 3 . If a nonlinear model of the phase locked loop is 
employed, it can be shown [4, chapter 4] that (8) is reasonably accurate for aj < 0.25. The 
Chernoff factor (21) can then be written as 

CW.) = exp (-ji- (fc, - ^ - i^)) (23) 

The value of p relates the phase jitter of a particular receiver to the channel signal-to- 
noise ratio. A smaller value of p gives a better receiver, since the phase noise variance <r| 
becomes smaller. 

Although the bit error probability Pb is the quantity of ultimate interest, a closely 
related and more readily determined quantity, the event error probability P e , will be used 
to characterize the performance of trellis codes. 

If x and x' are two symbol sequences corresponding to two paths through the trellis 
which are distinct for l branches starting at node j, and the decoder chooses the encoded 
sequence x' over the correct sequence x, this is called an error event of length / starting 
at node j. An error event starts where the two paths diverge and ends where the two 
paths remerge. A union bound on P e for a trellis code may be obtained by summing the 
probabilities of the error events of all possible lengths given a particular correct sequence 
x and averaging this quantity over all possible correct sequences x. 

With each incorrect path we may associate a sequence of incorrect trellis states S', 
while the sequence of correct states is denoted S r . Any error event of length l can then 
be described by l state pairs, (S 0 , S' 0 ), . . . ,(S,, S',), with S 0 = S' Q , S, = S'„ and S r ^ S' T 
for 0 < r < /, i.e., the incorrect path must not touch the correct path during the error 
event. Associated with these paths are the two symbol sequences x = (£o, £i, • • • and 
x' = . . . ,£*), where £ r ,£j. € A. The probability of an error event may be upper 

bounded using the Chernoff factors (with A = A^) between the individual signals of the 
two code sequences which the paths generate. We may therefore write 

Pr[(S 0 , • • • , S,) -> (S' , . . . , S',)} < C(x, x') = fl C(£ r ,4). (24) 

r=l 

3 See [4, p. 33 ff.] for graphs of the pdf of 0 for several loop signal-to-noise ratios. 
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Using (23) for the Chernoff factor we obtain 




( 25 ) 


n«p 


r= 1 


4JV 0 


, / |2 2 P K r 

\3Lv-3ir\ - 


1 


1 - pgj ) y/1 - pri 


= X r exp , 

where K, r — nt=i 3i- ~ independent of the noise power spectral density N 0 . 


(26) 


It becomes evident that the probability of choosing x' over the correct sequence x is 
determined by a generalized distance measure d(x,x') between the two sequences x and x', 
where 


i 


d(x,x') = J2 d (£r,x!r) = 2 ( l®r -£^| 2 - ~ 


l 

y 

r=l r= 1 

which after substituting for k t and g r becomes 




PVr 


(27) 


d(x ' x,) - i ~ ~ i - Kw ' -1 Iii5 


2p[^ r | 2 |^| 2 sin 2 (^ rr .) \ 

COs(^rr'))/ 


(28) 


Note that the distance measure (28) is an additive distance measure and reduces to the 
Euclidean distance for negligible phase noise, i.e., for p — ► 0. 

For MPSK, the generalized distance in (28) can be reduced further to obtain 


d(x,x') = ^2(1 - cos(<^ rr /)) - 


2ps in 2 ((ft rr >) ' 
cos(<^> rr -) / 


= E(2(l-cosW,,)) 1 _ 1 c ; s ^ r , ) ). 


(29) 


From (29) it is evident that the generalized distance is never negative for p < 0.5. It is 
also evident that close signals, i.e., small values of <f> rr >, suffer more from phase jitter than 
signals which are further apart. 
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Tables 1 and 2 show the normalized minimum free generalized distances of the best 
known 8-PSK and 16-QAM trellis codes, which were found by Ungerboeck [3]. The entries 
list the normalized minimum free generalized distance d(x,x') of the codes for various 
values of p. Note that the entry for p — 0 is the ordinary minimum free Euclidean distance 

dfree- 
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u5 

5 

45 

16 

34 

5.75 

5.6869 

5.3946 

5.0039 

3.6148 

u6 

6 

103 

030 

066 

6.34 

6.2686 

5.9296 

5.4866 

3.9308 

u7 

7 

277 

054 

122 

6.58 

6.5152 

6.2193 

5.8161 

4.2468 

u8 

8 

435 

072 

130 

7.52 

7.4141 

6.9997 

6.4522 

4.5629 

u9 

9 

1007 

164 

260 

7.52 

7.4141 

6.9997 

6.4522 

4.5629 

ulO 

10 

2003 

164 

770 

7.52 

7.4141 

6.9997 

6.4522 

4.5629 


Table 1: Ungerboeck’s 8-PSK codes on an AWGN-channel with phase jitter. 


i.d. 

V 

H(D)° 

H{Df 

h{d y 

P = o 

p = 0.01 

p = 0.05 

p = 0.1 

p = 0.25 

uq2 

= 2 

5 

2 

- 

1.6 

1.5709 

1.4468 

1.2727 

0.5714 

uq3 

3 

11 

02 

04 

2 

1.9671 

1.8339 

1.6599 

1.0159 

uq4 

4 

23 

04 

16 

2.4 

2.3598 

2.1959 

1.9833 

1.2041 

uq5 

5 

41 

06 

10 

2.4 

2.3599 

2.1967 

1.9864 

1.3099 

uq6 

6 

101 

016 

064 

2.8 

2.7526 

2.5588 

2.3067 

1.4597 

uq7 

7 

203 

014 

042 

3.2 

3.1436 

2.9209 

2.6269 

1.5416 

uq8 

8 

401 

056 

304 

3.2 

3.1453 

2.9209 

2.6269 

1.5416 

uq9 

9 

1001 

0346 

0510 

3.2 

3.1870 

3.1304 

2.9566 

1.7744 


Table 2: Ungerboeck’s 16-QAM codes on an AWGN-channel with phase jitter. 


4 Performance of TCM on Channels with Phase Jit- 
ter 

The performance of a code on an AWGN channel depends on its distance spectrum. The 
generalized distance measure introduced in the last section for channels with phase jitter 
changes this distance spectrum. Furthermore, for all but constant envelope constellations, 
the generalized branch distance d(x r ,2 l!r) is no longer reciprocal, i.e., d(& m ,a p ) ^ d(a p ,n m ) 
for some a m ,a p (E A. The distance spectrum can be efficiently computed only for quasi- 
regular codes [10]. Quasi-regularity is defined as follows. Let S and S' be two states in a 
trellis code and let the signal selector error vector e be the binary sum of a signal selector 
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v from state S and a signal selector v' from S', i.e, e = v ® v' . The distance polynomial 
Ps,s',e(x ) is then defined as the set of distances between signals leaving S and S' whose 
signal selectors differ by e. A trellis code is quasi-regular if and only if any non-zero distance 
polynomial Ps,s’, e depends only on e but not on S and S'. In [10] it is shown that all k/k+1 
trellis codes which are designed using Ungerboeck’s set partitioning [3] are quasi-regular. 

The next two lemmas establish that these codes are also quasi-regular with respect to 
the generalized distance measure introduced earlier. 

Lemma 1 8-PSK codes based on Ungerboeck’s set partitioning are quasi-regular with re- 
spect to the generalized distance of (29). 

Proof: The generalized distance is reciprocal for constant envelope schemes, which can 
be seen by inspecting (29). Thus, only the values of the actual distances between points 
are altered by using the generalized distance instead of the Euclidean distance and the 
quasi-regularity of these 8-PSK codes for the Euclidean distance is a sufficient condition 
for quasi-regularity for the generalized distance. Q.E.D. 

Lemma 2 16-QAM codes based on Ungerboeck’s set partitioning are quasi-regular with 
respect to the generalized distance of (28). 

Proof: Figure 3 shows the first level of the set partitioning of 16-QAM. The two sets 
are labeled B1 and BO in accordance with Ungerboeck’s notation. Codes based on set 
partitioning select signals either from set B1 or BO at each step, regardless of the encoder 
state. Four possible combinations of the trellis states S and S' can occur. Furthermore, e 6 
{0000,0010,0100,0110,1000,1010,1100,1110} only if (5,5') e Bl x Bl or (5,5') e B0 x 

BO. From Figure 3 it can be seen that inverting the last bit in the signal selectors axially 
reflects the set Bl into B0 and vice versa. This reflection will not affect the magnitude 
|<^ rr '| of the angle of the signals with respect to each other nor the vector lengths |x r | or 
Furthermore, (28) is an even function of the phase difference <f > TT /, and therefore Ps,s',e 
is independent of S and S'. 
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Subset Bl 


Subset BO 



Figure 3: The first level of set partitioning of 16 QAM. 


For e € { xxxl }, where x is an arbitrary bit, the trellis states ( S,S ') must either 6 
-Bl x BO or 6 BO x Bl. Again due to the special assignment of signal selectors, all 
distances generated between Bl and BO are simply reflected in BO and Bl, which leaves 
the mutual angle magnitudes and vector lengths unaffected. Thus 16- QAM with the signal 
assignment of Figure 3 is quasi-regular with respect to the generalized distance of (28). 

Q.E.D. 

Figure 4 shows the generalized distance spectrum of Ungerboeck’s 8-PSK code u4 for 
several values p. It is worth noting that the spectrum not only shifts, but that the lines are 
also shifted with respect to each other. Because the generalized distance is reciprocal for 
constant envelope schemes, no new lines show up. Note also that the minimum generalized 
distance decreases as p increases. 

Figure 5 shows the generalized distance spectrum of Ungerboeck’s 16-QAM code uq3. 
Unlike the 8-PSK code, each original spectral line breaks up into a set of lines due to 
the fact that the signals are no longer reciprocal. For p — 0.01, these clusters of lines 
can still be identified with the original spectral line, while for larger values of p a more 
continuous spectrum emerges. It becomes evident that the minimum generalized distance 
d(x, x'), while indicative of a code’s performance, is no longer the paramount performance 
parameter, but that a number of spectral lines influence the event error probability of the 
code. 
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Figure 5.2: Spectrum of the 16- QAM code uq3 for 5 different values of the phase-noise- 
to-channel-noise ratio p. 
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Figure 6 shows the error performance of the code u4 with various phase-noise-to-channel- 
noise ratios. The event error probability of this code has been calculated using a variant of 
the distance spectrum algorithm for quasi-regular codes presented in [10]. It is evident that 
even though phase noise causes a degradation in system performance, this degradation does 
not affect the error probability dramatically. Even for phase-noise-to-channel-noise ratios 
as large as p — 0.25, the degradation is only 1.5 - 2dB. (Note that for a signal-to-noise 
ratio of lOdB, p = 0.25 translates into a phase standard deviation of erg « 9°.) 

Figure 7 shows the error performance of the code u4 for several phase noise standard 
deviations erg. Since it is unreasonable to assume that the phase jitter is independent of the 
channel noise, these curves do not represent actual system performance but are included to 
demonstrate that for phase errors erg < 5° the degradations due to phase jitter are small. 
Even at a target error rate of P e = 10 -8 the degradation of a system with 5° phase jitter 
is less than ldB compared with ideal synchronization. For larger values of the phase jitter 
(e.g., erg = 10°), the error curve flattens out at P e « 10 -6 , indicating a maximum achievable 
error performance for non-ideal phase synchronization. 

Figure 8 shows the error performance of the code uq3 for various values of p. Most of 
the comments on 8-PSK coded modulation apply also to 16-QAM coded modulation. In 
general, the degradation suffered due to phase noise is larger however. 

Figure 9 shows the increase in S/N required to achieve an event error probability of 
P e = 10~ 6 plotted versus erg. The degradation curves for uncoded modulation are included 
for reference. It is worth*noting that the expansion of the signal set which is introduced 
in TCM does not severely affect the degradation due to phase jitter, i.e., the degradation 
of uncoded QPSK is about the same as the degradation of coded 8-PSK for small phase 
jitter angles erg. For larger phase jitter angles, coded 8-PSK is even more robust than 
uncoded QPSK and suffers significantly less degradation for eg > 5°. Note, however, that 
the degradation suffered by uncoded 8-PSK is much more severe. 
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Figure 9: Signal-to-noise ratio penalties due to carrier phase jitter at an event error 
probability of P e = 10 -6 for coded 8-PSK and 16-QAM modulation. 

5 Carrier Phase Noise 

In this section we show that the general method of analysis can be expanded to include 
other phenomena than channel noise which result in carrier phase jitter. As an example we 
discuss oscillator phase noise, which results because the receiver and transmitter oscillators 
will not be perfectly stable, but will exhibit phase fluctuations. These fluctuations result in 
carrier phase noise and are the principal cause of line broadening. They can be modeled as 
a white noise process [11]. The carrier is then cos(a> 0 £ + Z(t )), where Z(t ) is an integrated 
white Gaussian noise process, i.e., the frequency noise Z(t) is closely modeled as white 
Gaussian noise with one-sided noise power spectral density N z . The phase noise itself is 
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represented as 

Z(t ) = 2 ir f Z(t)<1t. (30) 

J oo 

Denote the Fourier transform of Z(t) by f(Z(t)) = Z(u>). It can be calculated from the 
Fourier transform of p(Z(t)) = \/W z as 


Z(u) = 27r 


ju 


+ 7tZ(0)6(u>) 


(31) 


where we may discard the DC-component ttZ(0)<5(u;) since it is tracked by the phase-locked 
loop, and we obtain for the power spectrum of Z{t) 


S z {u) = \Z(u)\ 2 = 4tt 2 ^. (32) 

From (32) we see that the phase noise Z(t) is a correlated Gaussian process. The phase 
noise variance a\ that results from the carrier frequency fluctuations can then be evaluated 
as [4, p. 27] 

2 i r N * 

2tt JikBl u> 2 Bl 

where Bl is the phase-locked loop bandwidth encountered earlier and we have assumed an 
ideal low-pass filter in the phase locked loop with bandwidth Bl- 

Using a linear model for the phase locked loop, the two independent Gaussian contri- 
butions to the carrier phase jitter process result in a total Gaussian process, and we obtain 
for the carrier phase variance crj 



a 9 — pN o + 

N 

= B l N 0 + 

Bl 

= pN 0 + (34) 

P 

It is evident that increasing the loop bandwidth Bl decreases the variance contribution <j 2 
from the carrier phase noise because higher frequencies can be tracked, while it increases 
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the portion of the variance induced by channel noise. Decreasing the loop bandwidth has 
the opposite effect. 

Equation (34) for the phase noise variance may now be used in (18) to evaluate the 
Chernoff factor, and we obtain 


C(x r ,&, A) = 


0 * 2-2 2 
2 A 


„ e 1 - 2 ^ 


t - 2A<t|7 ] r 


= e -A(l-A^ 0 )|i r -^P 


2A 2 (pAf n + Nz/p)"? 
g 1— 2A(pNo + N./p)i7r 



(35) 


This expression for the Chernoff factor may be used in the algorithm described in [10] 
with A = Arq to evaluate the event error probability P e of TCM schemes. Figure 10 shows 
the error performance of the code u4 with a noisy carrier and N z = 1.5 x 10 -4 [rad 2 /s]. Note 
that the error curves for different values of p cross. From (34) it can be seen that choosing a 
small p suppresses channel noise induced phase jitter while increasing the portion induced 
by phase noise. Thus for low signal-to-noise ratios a receiver with p — 0.01 outperforms a 
receiver with p = 0.25, while for high signal-to-noise ratios, where the phase noise portion 
is dominant, the situation is reversed, leading to the crossing of the error curves. 

Figure 11 shows the error performance of code u4 for a high value of the carrier phase 
noise, i.e., N z = 2.5 x 10 -3 . The curves for p = 0.01 and p = 0.05 demonstrate that there 
exists an irreducible error probability which is due to the phase noise of the carrier and 
which does not decrease with the channel S/N. This is manifested in the error curves’ 
flattening out for large values of Es/Nq, while the performance curves for no carrier phase 
noise show an exponential decrease with Es/Nq (Figure 6). 

The results in this section are of an approximate and illustrative nature. More precise 
expressions for the phase jitter variance in (34) should be substituted for a detailed system 
performance analysis. 


6 Conclusions 

We have analyzed the performance of TCM schemes in the presence of phase jitter. The 
performance degradation due to phase jitter was calculated using the generalized distance 
spectrum of a code. In the case of practical coherence, when the phase jitter is induced only 
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by channel noise, the performance degradation was found to be moderate, even for relatively 
large values of the phase-noise-to-channel-noise ratio />. Other sources of phase jitter, such 
as oscillator phase noise, were found to result in more severe performance degradation. 
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Figure 10: Event error probability of coded 8-PSK modulation (code u4) for various 
values of the phase-noise-to-channel-noise ratio p in the presence of carrier 
phase noise with value N z = 1.5 X 10~ 4 . 
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Figure 11: Event error probability of coded 8-PSK modulation (code u4) for various 
values of the phase-noise-to-channel-noise ratio p in the presence of carrier 
phase noise with value N z = 2.5 x 10 -3 . 
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Appendix A 


In this appendix we present the use of an adaptive phase predictor to whiten the residual 
phase errors. The carrier is tracked by a carrier recovery loop which produces the local 
reference signal cos(u>ot 4- 9(t)), where 9(t) is the carrier recovery loop phase error. At the 
sampling instants nT, this phase error will give rise to phase offset angles 9 r . These phase 
offset angles are approximated by a Gaussian process [2,4]. Since the recovery loop has a 
small bandwidth, the phase error samples will be strongly correlated. 

The method of achieving independent phase offset angles presented in this appendix is 
linear prediction. We will use a linear predictor driven by N previous symbol decisions to 
predict the phase offset 0 T of the r-th sampling instant, as illustrated in Figure Al. The 
iV-th order predictor is fed by the measured phase offset O'- of the j-th received symbol 
Lj with respect to the j-th decoded symbol &,•, where Q.j is the best estimate of the j-th 
transmitted symbol CLj which is taken from the survivor with the best metric. The r-th 
phase offset 9' r is then linearly predicted as 


= (A.i) 

:=i 

Note that the measured phase offset 0' is given by 

Oj = 9j + f3j, (A. 2) 

where 9j is the ^’-th phase offset of the recovery loop and flj is a phase offset induced by 
the additive Gaussian noise affecting the j - th received signal point y _ ., as shown in Figure 
A2. 

We are using the predicted phase offset 9' r at time unit r to compensate for the actual 
offset 9 t . The received signal point y_ r is digitally rotated by —9' r into y*. 
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Figure Al: Phase recovery circuit with phase predictor. 


The offset angles 0j are independent because they are caused by channel noise, which is 
white, and we may evaluate the autocorrelation function of as 


E 



E [(^j + Pj)(9j-k + Pj-k)\ 


— E [OjQj-k] ■+■ E 


= R ee (k) + 6(k)N 0 , 


(A.3) 


where R$e(k ) is the autocorrelation function of the phase errors Oj due to the phase recovery 
loop and 8{k) is the Dirac delta function. The cross terms in the above equation vanish 
due to the assumed independence of the offset angles /?_,• and the phase errors 6 r A simple 
geometric consideration shows that Np « where Es is the signal energy (see [4]). 
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Figure A2: Combined effect of phase jitter and channel noise on the received signal point 

h- 

The linear predictor (Al) will predict 9' r using the optimum predictor coefficients h = 
[fti, ft 2 , • • • , M T , which are given as [6] 


where 


h = R -1 r, 


R = 


-Rm(O) + Np R$e( 1) 

Ree{ 1) Ree{ 0) + Np 

Reo{ 2) RgeiX) 


Ree(N-l) - 
Ree(N - 2) 
Reo(N - 3) 


L R 66 (N- 1) Ree(N — 2) 


R$e{ 0) + Np J 


(A.4) 


(A.5) 


is the correlation matrix of the phase error process 0' and r = [i2w(l), R$g( 2), • • • , Reff(N)] T . 
A more detailed discussion on linear predictors is contained in [6]. 

The digital rotation of y_ r leaves us with a residual phase error ip T . We are then interested 
in the autocorrelation function of ip r . Let us first consider the autocorrelation function of 
the total phase error process, i.e., of the difference between the full phase error 0' and the 
predicted phase error 0', which is calculated as 
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E [(0, - # r )(<>r-k - C*)] = 


= E 


N 


{Or + &)-£ hj(9 r .j + fir-i) (Or-k + Pr-k) ~ £ hi(6 r -i-k + Pr-i-k) 


i = i 


N 

£ 

i=l 


= E [0 r O r _ k ] + E [PrPr-k] ~ £ h { E [{O r + Pr)Or-i-k) ~ £ h,E [(0 r + Pr)Pr-i-k] 

t = 1 1 = 1 

— £ hjE [0 r -j{0 T -k + Pr-k)] — £ [^r-j(^r-fc + Pr-kj] + 

j=l j=l 

E E + &-,)(«_(.» + 

j = l 1 = 1 

AT AT 

= 12m(*) + 6{k)N 0 - £ W2 M (t + k) - h k N 0 - £ hjR ee (k - j) 

t=i i=x 


+ £ hi £ hj (Ree(k + i - j) + 6{k + i - j)^j 
= 0; 1 < \k\ < N. 


(A.6) 


The residual phase error process is a white Gaussian process with variance 

Etor- KM - K)\ = Ng + fl w (0) - E kjRuU) 

i=i 

= N 0 + a 2 e -jrh 3 R de (j). (A. 7) 

j=i 

We are free to interpret the phase error process as the result of two independent processes. 
The first process is channel noise and results in a phase error with variance N 0 . The second 
process is the residual phase error process with variance cr| — hjRee(j)- It is a white 
Gaussian process as required in section 1. 
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Abstract 

Bandwidth efficient data transmission using trellis coded modulation (TCM) can significantly increase a 
digital communication link’s reliability without sacrificing spectral efficiency. In this paper we discuss 
TCM on channels with impulse noise. The intentional partial band jammer with optimized duty cycle is 
analyzed analytically and taken as a worst case impulse noise interferer. It is shown that a new code design 
criterion, the effective length of a cod6 becomes the central code parameter determining the performance 
of TCM. A number of new TCM codes are presented and their performance is calculated. It is shown that 
these codes fare significantly better than codes designed for additive white Gaussian noise channels. 

1 Introduction 

Practical communication channels are often not well modelled by additive white Gaussian noise (AWGN) 
channels. Yet most performance analyses of trellis coded modulation (TCM) have focussed on AWGN 
channels. In this paper we discuss a general class of impulse noise channels. Such impulse noise channels 
arise in various situation in practice. Man made environmental noise such as that caused by automotive 
ignition, neon lights or the switching of power machinery is impulsive. Intentional pulsed jamming, which 
is taken as the worst case impulse interference, also generates nonuniform noise. The most interesting appli- 
cation, however, is multiple access of a particular frequency band (FDMA) as envisioned in modem cellular 
radio. One proposed mechanism uses frequency hopping to overcome the often poor transmission quality 
of mobile radio channels. In such a situation, the probability that a particular user jumps to a frequency also 
claimed by another user is not zero. This is called a hit, and we denote the probability of a hit by p. An 
analogous situation arises in the case of time domain multiple access (TDMA). 

As worst case situation we may view the interfering simultaneous users of a frequency band as a jammer 
trying to disrupt communication. We assume throughout the derivation that the channel symbols are inter- 
leaved, so as to assure independence of the jammer state at each time interval. The jammer shall be average 
power limited at the jamming power Pj. In a partial band jamming situation (FDMA), this jamming power 
is spread over a portion p of the total frequency band at a power spectral density Nj/p, where Nj is the 
power spectral density if the total jamming power Pj is spread over the entire frequency band. In a TDMA 
situation, the jammer will be pulsed during a fraction p of the time at a power spectral density Nj/p. In 
either case, for a user the jammer assumes two distinct modes of operation, it is on during a fraction p of 
the time at a power level Nj / p , and off the rest of the time 1 — p, where p € [ 0 , 1] is the jammer duty 
cycle. We further assume at this point that the decoder is furnished with perfect side information, i.e., at 
each symbol interval r the jammer state, s T £ {on,off}, is known. 

A TCM communication system (Figure 1) consists of a trellis encoder, a signal interleaver, the trans- 
mission channel, a signal deinterleaver, and a trellis decoder. A rate R = k/n trellis code is generated by a 
binary convolutional encoder followed by a mapper. The convolutional encoder is a finite state automaton 

1 This work was supported by NASA Gram NAG 5-557. 


with 2" possible states, where v is the memory order of the encoder. At each time interval r, the encoder 
accepts k binary input bits ( u*, ,u‘) and makes a transition from its state S r at time r to one of 2 1 

possible successor states S r + 1 . The n = n— (k — ~k) output bits of the convolutional encoder and the k — k 
uncoded information bits ( u*, . . . , u* +1 ) form one of 2* binary n-tuples v r = ( t/“, , . . . , vj ) , which is 

translated by the mapper into one of A = 2 “ channel signals from a signal set A = {fii , az . • • • » fu}. The 
uncoded information bits do not affect the state of the convolutional encoder and cause 2 k ~ l parallel tran- 
sitions between the encoder states S r and S r +i- A rate R = k/n trellis code transmits k bits/channel signal, 
where the channel signal set contains A = 2 B signals. If such a TCM communication system replaces an 
uncoded system that uses a signal set with A 1 = 2* signals, the overall transmission rate is preserved, and 
we call such a TCM system bandwidth efficient Due to their practical importance, we restrict attention to 
trellis codes for 2-dimensional signal sets. 



Figure 1: Trellis coded communication system. 

The interleaver/deinterleaver converts the channel to a memoryless channel and insures that the signals 
in the received sequence are independent They are decoded by a sequence estimator (usually using the 
Viterbi algorithm). The Viterbi algorithm finds the signal sequence that most closely corresponds to the 
sequence of received signals. It achieves this by calculating a decoding metric m(x, y) between x and y, 
where x = ( 3Li , • . • , it) is a possible sequence of 2 -dimensional transmitted signals and y = ( y j , • . • , yj 
is the received sequence. m(x,y) is some non-negative function of x given y, which is inversely related 
to the conditional probability that x was transmitted if y was received. The decoder will then choose the 
message sequence x for which this metric is minimized. It makes an error if it decodes a sequence x', given 
that the correct sequence, i.e., the transmitted sequence, was x . This will happen if m(x', y ) < m( x , y) . 
In the case of channel state side information the decoder will use m(x, b ,y) as its metric, where b is the 
side information obtained from the channel. 

2 Optimum Receiver for Impulse Noise Channels 

In this section we will discuss the optimum receiver strategy if the channel’s noise disturbance can be mod- 
elled as impulse noise as defined in the introduction. In [Wozencraft-Jacobs, 1965], the optimum receiver is 
defined as the receiver using the maximum a posteriori (MAP) probabilities as the decision criteria. In the 




case of equally likely messages - the case we will discuss - this optimum receiver becomes the maximum- 
likelihood (ML) receiver. The MAP-receiver chooses the message x k if and only if 

Pr(x* |y) <Pr(Xj |y); for* = 1, . • • , M,i i k, ( 1 ) 

where M is the number of messages. Using Bayes’ Rule, (1) is equivalent to choosing the message x k 
for which Pr(y | x k ) is largest. As the transmitted signals are interleaved the noise in our channel is a 
non-uniform non-coirelated Gaussian process and Pr(y | x) is given by 


Pr(y | x) 




where the channel noise power spectral density JV*. is given by 


( 2 ) 


Nh = Nj/p + No with probability p ( 3 r = on) 

Ncr ~ 1 (3) 

N g = No with probability 1 — p (s r = off) . 


and N 0 is the channels thermal noise power spectral density. If Nj - 0 or p = 1, we have the ordinary 
AWGN channel. Selecting the message x* which maximizes (2) is equivalent to selecting the message that 
minimizes the metric 



because XlUo ^ 7 ^ and No are independent of the particular message sent The metric in (4) is the sum 
of individual branch metrics and the optimum receiver will employ the maximum-likelihood branch metric 
c(s r ) ly, — £ r | 2 , where 



Nj^No ^ S T - On 

1 if s r = off . 


(5) 


Note that if the jamming power spectral density Nj » No, the receiver will give a very low metric to any 
symbol received while the jammer is active, that is it will mark those symbols as unreliable. 


3 Error Bounds 

Because it is unfeasible to exactly calculate the error performance even on a AWGN channel, we resort to 
bounding techniques. Denote the two code word error probability when x is sent and x' is decoded, given 
the jammer state sequence s = ( si , S 2 , . . . , sj) by P( x -* x' | s) . 

The two code word error probability can then be expressed in terms of the Chemoff bound, i.e.. 


= f[P(x^x')] 

< E m inP J^exp(— Ac(a r ) ((ly, — a£| 2 — I& — lr| 2 )) 

* 1 Lr= 1 J . 


P(x — ► x' | s) 



( 6 ) 


< mmS^j^nexpC-^cCsr) ((|^.-4l 2 -l^-irl 2 )) 

= min^[lJcxp (-Xc(s r ) ((£? -£?) + 2 y/a,-^))) 

* 1 Lr»l 

= minC(x,x',X). 

As the transmitted symbols are interleaved, the received symbols are independent from one another. 
Noting that y = z, + Hr. where is 2-dimensional Gaussian noise with one-sided noise power spectral 
density given by (3), and applying the second law of expectation to the above equation lets us move the 
average over y into the product, i.e.. 


C(x 


,x',X) = El[ 

lr*l 


exp(-Xc(5 r )(i,.-a^) 2 )£? {exp ( -2 Xc(s r ) -a£))} 

Her 


= E m exp(-Xc(s r )(a, -xH.) 2 )exp(\ 2 c 2 (sr)Ner(x r -z! r ) 2 ) 


Lr=l 


(7) 


Rewriting the above equation as 


C(x,x',X) * E 


r i 


IIcxp (-Xc(s r )(Xr - Zr) 2 ( 1 - \c(Sr)N„)) 


Lr»l 


( 8 ) 


and observing that c(s r )N„ = No is independent of the jammer state, we note that (8) is minimized inde- 
pendently of r by setting X = X mtn = 1/2 No, i.e., 


C(x,x',X TO , B ) 


< 



(a r -a£) 2 V 

IN* )' • 


(9) 


This Chemoff bound breaks up into a product over the individual jammer states s T , which we call Chernoff 
factors, i.e., 


i 


C(x,x',\ min ) *IJC(i r ,x') « 

r* 1 



(Xr-3^) 2 Y 
4Ncr ), ' 


(10) 


Due to the interleaving process, the jammer states s r are statistically independent and we may average over 
s, that is, we may drop the dummy subscript r as the jammer states are independent identically distributed 
(i.i.d) . (Note that without interleaving we could still proceed further analytically if the jammer state se- 
quence can be described by a Markov chain, see [Gallager, 1968]). The Chemoff factors are then given by 


C(x,x\x mi j = nc(x,, *;) = !!£ 


exp 


]C(x r ,x ' T ) = J] 

el r=l 

The cutoff-rate of the channel, Ro can be calculated [Schlegel, 1988] as, 

A A 

:i 

*1 p*l 


(Xr-3^) 2 y 

Ncr ). 


(ID 


Ro = -iog 2 ££p(am)p(ap)C(am,ap) 


= - lo g2 [EEp^™)p(V^ 

\m»l p*l 




( 12 ) 


where the Chemoff factor between a m and Op, i-e.. 


CCibn.flp) 


E 


exp(- 


( S.m flp) ■, 

4 Ncr } 


= pe'inwTZ + (l ~p) e w“. (13) 

An enemy jammer will try to minimize the cutoff rate by varying p, as this will raise the error probability 
on the average for all coding systems. Rather than minimizing (12) for one particular constellation we will 
introduce an upper bound on (12), which is exact for the simplex constellation, and then minimize this upper 
bound. This will provide a robust minimizing p which is valid for any constellation. 

Let us now proceed by substituting (13) into (12) to obtain 


Ro - — log 2 



<io.-y 2 

pe MN 0 .Nj/,) + (i _p) e *"o 


(14) 


Defining b = £4 x p(o,-) 2 , we may rewrite the above expression as 


2~ito _ 


min 

Pi 


^(l-i)SE g ( - fl, "~ P(flp) 


m=l p=l 
tn^p 


1-6 


pe mnPnJ7T) + ( i _ p) e **o 


A 


(15) 


Introducing the factor ( 1 — 6) in the second term above makes it an expectation. As the exponential function 
is U-convex we may apply Jensen’s inequality to the second term to obtain 


2-* > minf 6 + ( 1 — 6) 


pexp 


££.1 Z£lP(flm)p(flp)(am-gp) 2 

4(JVo + JV}/p)(l-6) 


+ ( 1 - p) exp 


Ei-i E^i p(a TO )p(a P )(flm - lip) 

(4AT 0 )(l-6) 



(16) 


As an optimal signal set has its centroid at the origin, i.e., p(a<)a» = 0 • (sec [Wozencraft- Jacobs, page 

248] or [Massey, 1974]), we identify the above summations as 


X)52p(flm)p(a P )(flm -a P ) 2 = 2 E s , 


m=l p« 1 


(17) 


where Es - £[a?], the average signal energy. Substituting into (16) we obtain 

2~ Bo > min^6 + (1 -6) |pe“w?lw“-*» + (1 - p)e -(4 "o^-» . (18) 

Using Lagrange’s method it can be shown that the minimizing distribution in the above lower bound is the 
uniform distribution p, = 1 /A. Using this we may further simplify the bound to obtain 

2~A> > 1 + ^l _ (pe'^J^r : ttx + ( l _ p) e -u^-m^j . (19) 



This equation is an upper bound for any constellation and equality holds for the simplex signal set for which 
( a m — a p ) 2 is constant for all m, p such that m ^ p. 

An enemy jammer will now try to minimize Ro for any constellation. It will therefore maximize the 
right hand side of (19) and evaluate p such that 


JL ♦*,/*»< i-i am + ( 1 — = 

(■-»{ 


c _ «W 0 *Wy/p)( 1-1/A) 1 — p 


2E s Nj 


4(pN 0 + Nj) 2 (l - 1/A) 


_ e 


H 


( 20 ) 


As the jammer will not have precise knowledge of the noise power density No at the receiver, and as Nj » 
No , it seems sensible to neglect No in the above equation and set 




e ~ I pH 1 - i /A) M _ p 


2 E s Nj 


4(piVo + Nj) 2 ( 1 - 1/A) 


2g? 

_ g _ «w 0 <i-VA) 


pfl.g 


» e 1 - p 




2JV>(1-1/A) 




( 21 ) 


This maximum is achieved by setting 


P = 


Es 


1 


for Es/Nj >2(1 - 1/A) 
otherwise. 


(22) 


Equation (22) reveals that a smart jammer will shorten the jamming duty cycle as the signal energy increases 
in the transmission to be disturbed. Substituting the maximizing p into (13) we obtain for the Chemoff factor 
between x. r and xl 


C(z r ,x!r ) = 


2^(1 -1/A) 

Es 




o- t ^> 


(*- 






V 


(23) 


Even though it seems that this Chemoff factor exhibits an exponential characteristic, this is not the case, 
which is shown by the following argument. Let d 2 ^ be the normalized distance between the signals x T and 
i.e., d 2 ^ is independent of the signal energy Es and is defined as = (x, — x!r) z /Es • We may now 
write 


C(XrX) = 2Mj(1 Es — e + (l - (24) 


As we are primarily dealing with a jammed channel (i.e., Nj » No as stipulated earlier), we will further 
assume that Es » No, as otherwise Es < Nj and then the transmitter would not be successful in sending 
any information. We now obtain 


2W1-1/A) / 2JVj(l — 1/A) \ 

E s e<- , '-' /A,r - ( Es) 


C(Xr,x!r ) 


(25) 


Note that 


C(£ r ,ai.) 


{ Z(l- 


2(1-1 M) 


—\/A)/2 Es 
1 


ik " Krr'jfc if ir 7^ 


iflr = 4- 


(26) 


where is a constant that depends on the signals used, but not on the signal to noise ratio Es/Nj. 

It is interesting to note that the jammer achieves a degradation of the Chemoff bound in the sense that 


C(x T , si,) is no longer an exponential bound of the form e~ Krrt but is only inversely proportional to the 
signal to noise ratio Es/Nj as evidenced in (26). 

Using (26) in (12) we obtain for the cutoff-rate Rq 


The upper bound shows clearly that the error probability no longer decreases exponentially but only poly- 
nomial^ with the signal to noise ratio Es/Nj. This will be of importance to the code design for jammed 
channels. 

Figure 2 shows the cutoff rate for some popular signal constellations using the minimizing value of the 
duty cycle p, as well as the upper bound on Ro, i.e., (27) for A = 5. It is worth noting that the simplex 
bound (drawn for A = 5 ) is less than 1 dB off from the two 32-point signal constellations shown in Figure 
2. Another interesting observation is that the cutoff rate for the rectangular and the constant envelope signal 
constellations practically coincide, i.e., the signaling advantage, which, say 16-QAM has over 16-PSK on a 
AWGN-channel has disappeared. 

Figure 3 shows the cutoff-rate for 8-PSK with the duty cycle p as a parameter. If the receiver is capable 
of extracting the jammer state side information - which it can achieve by measuring the ambient noise if 
Nj /No is large - the jammer gains nothing by shortening its duty cycle for Es/Nj < 10 dB. Because 
the receiver knows the current jammer state it will simply blank out the jammed signals and have enough 
redundancy to recover the message. For higher values of Es/Nj the jammer achieves some degradation 
of the communication link by shortening its duty cycle in accordance with (22). The achieved degradation 
is small however compared to uniform noise jamming (p = 1), if the receiver has information about the 
jammer state. 

If the receiver is unable to extract side information from the received signal, i.e., the present jammer 
state s r is not known, it will have to use the ordinary Gaussian correlation metric 


l/A) ( 2N,(l-l/A) \ 

( E s j 


l 

™(x,y) = 


r=0 


and the Chemoff bound (8) becomes 


r t 


n« 

r=l 


-A<Ir-£) 2 a-A*«r) 


C(x,x',\) = mm f?| 

This expression cannot be minimized analytically. However the Chemoff factors are given by 

C(x r ,x' r ,\) = 


= pg-M^-j^a-MAO/zH-Afo)) + ( i _ p ) e - x <tr- & 2 (i-W) 
- + 1 - p) . 


(28) 


(29) 


(30) 



Substituting (30) into the formula for the cutoff-rate Ro we ob tain 

A a 

Ro = -log 2 min 5353p( a jp(a P ) c (a™,ap,X) 

* m=l pal 

= - log 2 min | £ J2p(a m )p(iip)e~ > ' (l '~^ )2( - 1 - XNo) + 1 _ p 

* \m*l p=l 



(31) 

This expression for Rq is a function of the jammer duty cycle p. 


Rq [bits/symbol] 



Figure 2: Cutoff-rate Ro for some popular signal constellations and the simplex upper bound, p is chosen so as to 
minimize f2o. 
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Figure 3: Cutoff-rate of 8-PSK for various values of the jammer duty cycle p. 


Figure 4 shows Ro without side information for a jamming power-to-noise ratio Nj/No = 100 . If the 
receiver has no jammer state side information, shortening the duty cycle p is detrimental at all values of 
Es/Nj. In fact there is no lower limit for p. A similar behavior of the cutoff-rate has been observed for 
the binary channel in [Omura-Levitt, 1982]. In practice, however, the decoder would modify the Gaussian 
correlation metric by limiting the maximum values which can assume. This clipping somewhat 

reduces the catastrophic effect of a very short jammer duty cycle p , see also [El-Wailly-Costello, 1975]. 
It is evident, nonetheless, that obtaining side information about the jammer state is crucial to successful 
communication over a jammed channel. 



Rq [bits/symbol] 



ft™ 


Figure 4: Cutoff-rate Ro for 8-PSK for the jammed channel without jammer state side information. 


4 Trellis Coded Modulation 

The event error probability of TCM can be upper bounded by the transfer function bound [Schlegel, 1988] 

P'<T, A < P <’ < 32) 

t 

where the sum is over all code sequence pairs x , x' whose two code word error probability is overbounded 
by the bound Pt, where Pt = nt=i C(x r , x' r ) and At is the average number of code sequence pairs with 
Chemoff bound P t . 

Among the parallel transition error possibilities there will be a dominant term. Pc, given by 


Pc = C(2r,lt) 


2(1 — 1 /A) Nj 

gA*( 1 —\/A)/2 Eg’ 


(33) 



where A 2 is the smallest normalized Euclidean distance between signals on parallel transitions. From this 
it is evident that parallel transitions severely degrade code performance because P c varies only inversely 
proportional with Es/Nj. 

For trellis error events, the dominant term, Pi, is given by 

Pi = JlCiZrX) = (2d - 1/A)|^exp(-2(1 - 1/A)<£(x,x')), (34) 

where d(x , x') = £t=o is the normalized Euclidean distance of the signals (x , x') on the correct and 
incorrect path. In this case, the dominant code parameter is the effective length l' m = min( l') of the code, 
where l' is the length of two code sequences less the number of time periods where they both are assigned 
identical signals. A trellis code designed for jammed channels must have a large effective length and a large 
Euclidean distance on the path pairs having l 1 = l' m . Note that the minimum free Euclidean distance d/ ree , 
the dominant parameter for trellis codes on Gaussian channels, is of minor importance for trellis codes used 
on jammed channels. This is so because the path pair (x,x') that achieves df Ttt might have an effective 
length l' > l' m . Table 1 shows the effective length and the Euclidean distances of Ungerboeck’s 8-PSK 
codes. 


i.d 

V 

H{D)° 

B(D )> 

H(D)* 

c 

4(x,x') 

dfnt 

u2 

2 

5 

2 

- 

1 

4 

4 

u4 

4 

23 

04 

16 

3 

6 

5.17 

u5 

5 

45 

16 

34 

2 

6 

5.75 

u6 

6 

103 

030 

066 

3 

8 

6.34 

u7 

7 

277 

054 

122 

4 

6.58 

6.58 

u8 

8 

435 

072 

130 

3 

8 

152 

u9 

9 

1007 

164 

260 

3 

8 

152 

ulO 

10 

2003 

164 

770 

4 

10 

152 


Table 1: Ungerboeck’s 8-PSK codes on the jammed channel. The connector polynomials H(D)> are defined in 
[Ungerboeck, 1982]. 


Table 2 shows a list of 8-PSK codes designed for the jammed channel. The code design method is discussed 
in [Schlegel, 1988]. 
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23 
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15 

5 

43 

14 

36 

3 

8 

5.17 

j6 

6 
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036 
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4 

8 
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P 

7 

223 

076 

314 

4 

8 

4.93 

J8 

8 

673 

336 

164 

5 

9.14 

634 


9 

1413 

756 

244 

5 

10.58 

6.34 

jlO 

10 

3303 

1676 

504 

5 

14 

634 

jll 

11 

6403 

3436 

1264 

6 

11.17 

732 


Table 2: 8-PSK codes designed for the jammed channel. 


Figures 5-7 show the transfer function bound (32) on the error event probability of some interesting 
codes on jammed channels. The bound was evaluated using the methods discussed in [Schlegel-Costello, 
1988]. The codes are u5, u8, j5 and j8. Note that even though the codes j5 and j8 have a smaller minimum 
free Euclidean distance df Tee than their Gaussian counterparts, their effective length and their minimum 
Euclidean distance d(x,x') for path pairs with effective length l' = l' m is larger, d(x,x') = 8 for j5 versus 
<2(x ( x') = 6 foru5 andd(x,x') = 9.14 for j8 versus d(x.x') = 8 foru8. 


Figure 5 shows the event error probability of these four selected trellis codes on a most effectively 
jammed channel with jammer state side information, i.e., the jammer duty cycle p is given by (22). The 
special codes considerably outperform the Gaussian codes. At an error level of P, = 10 ~ 6 for example, 
j8 gains 5 .5 dB over u8. At P e ~ 10 j8 gains 2 .8 dB over u8 and j5 gains 4.5 dB over u5. It becomes 
evident that for this jammed channel df Tte is of secondary importance. u8 and j5 have the same effective 
length l' m = 3 and d(x,x') = 8, so their asymptotic error curves have the same slope. 


Error Event Probability P e 



Figure 5: Performance of trellis codes on a most effectively jammed channel with channel state side information 
available at the receiver. 


Figure 6 shows the event error probabilities for the four trellis codes on a jammed channel with duty 
cycle p - 0 .1 and no jammer state side information. Note that the jamming noise power is only 10 times 
the power used by a uniform jammer. The error curves therefore show a steeper slope which is typical for 
Gaussian noise. It is interesting to note however that j8 and j5 still have a better error performance than u8 
and u5, emphasizing the importance of a good effective length even in moderate impulse noise. 
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Figure 6: Performance of trellis codes on a jammed channel with jammer duty cycle p = 0 .1 and without channel 
state side information. 



If the jammer further shortens the duty cycle p , the performance of trellis codes without jammer state 
side information can be degraded almost arbitrarily as illustrated in Figure 7 for p = 0 .005 . The value of p 
characterizes severe impulse noise which not only flattens the error curves but also translates them along the 
Es/Nj axis. Note that the Es/Nj scale in Figure 7 is shifted by 10 dB compared to the previous figures. 
The performance difference between j8 and u8 and j5 and u5 has also increased. It becomes evident in any 
case that side information and a good effective length l' m of the code are essential for reliable communication 
on channels with impulse noise. 


Error Event Probability P e 



Figure 7: Performance of trellis codes on a jammed channel with jammer duty cycle p = 0 .005 and without 
channel state side infoimadon. 


5 Conclusions 


We have analyzed the performance of TCM on impulse noise channels. For the worst case interferer, the 
intentional non-uniform jammer, the performance of a trellis code is dominated by its effective length. We 
showed through an error bound analysis that the effective lenght retains its importance even for moderate 
impulse noise. A number of codes were presented that perform better on impulse noise channels than their 
corresponding codes which were designed for AWGN channels. 
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A CONSTRUCTION ALGORITHM FOR TRELLIS CODES 
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Abstract: A fast construction algorithm for good convolutional and trellis codes is presented. The 
technique can be used with any practical constraint length (up to about 30 ), any signal constellation, 
and any code rate. The construction does not require an exhaustive search, so that the complexity of 
the construction does not increase significantly with the constraint length. The codes constructed have 
slightly sub-optimum or optimum free distance. 


Introduction: A trellis code, or trellis coded modulation (TCM), consists of a convolutional 
encoder followed by a mapper. Figure 1 shows a typical trellis code as originally designed by 
Ungerboeck [1][2]. An encoder state is characterized by the values of the past information bits 
stored in the shift registers of the convolutional encoder. The incoming information bits determine 
the transitions or branches from one state to the other. Subset selectors depend on the incoming 
bits and on v past information bits only, where v is the constraint length of the code. The mapper 
transforms the subset selector into a subset of signals and uncoded information bits select one par- 
ticular signal from this subset. The free distance dj ree is the minimum squared Euclidean distance 
between any two code words (sequences of signals). 

There exist very few methods for constructing good convolutional or trellis codes [3][4]. 
Traditionally, short constraint length codes have been found by exhaustive searches. For larger 
constraint lengths, when the complexity of exhaustive searches becomes overwhelming, the 
searches are performed over restricted sets of codes. The algorithm presented in this paper 
searches over very small sets of codes (about 10 codes). 

Most construction techniques and search algorithms can be used on expurgated sets of 
codes [5], where the expurgation guarantees a good distance growth from both ends of error 
events. Expurgated bounds on the free distance of trellis codes guarantee that such sets of codes 
contain codes with a large free distance [5]. This is a consequence of the observation that good 
convolutional or trellis codes must have good distance profiles from both ends of error events. 
Note that, like quick-look-in and complementary codes [31(4], Ungerboeck codes belong to expur- 
gated sets of codes that achieve a maximum minimum distance between branches that reach and 
leave the same state [1]. 

Henceforth, we consider partition chains of the signal set which are composed of two-way 
partitions [5]. For simplicity, we suppose that the trellis has no uncoded bits (if the code has 
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uncoded bits, then signals can be replaced with subsets of signals). Figure 2 shows a trellis code 
with maximum growth of the distance along die first q and last q branches of error events where 
q — \n!k\. The double line represents k parallel input bits and each memory cell represents k 
parallel memory cells also called a column of the encoder. Each set of k outputs selects k levels 
of the partition chain. Note that most of the best known trellis codes have the structure shown in 
Figure 2 [1] and [6]-[8]. 

If 5 y is the minimum distance between signals in the same subset at level j in the partition 
chain (1 < y < n), and if 8 0 ^ < ... < 8„, then the free distance of the codes shown in Figure 

2 satisfies 

djree — 28 \ _ £ + 28 * _ 2k +•••+ 28 * _ 

The codes are constructed from both ends simultaneously. The beginning of the code is 
constructed column by column, and each column is constructed from top to bottom. Similarly, the 
end of the code is constructed column by column, but each column is constructed from bottom to 
top [5]. If the code is large enough, q columns are constructed at both ends of the code. If the 
constraint length of the code is not large enough to accommodate 2q columns, then the columns 
are constructed in the following order: first the first and last columns, then the next < 7-1 left 
columns, and finally the next q - 1 right columns. The construction stops when the desired con- 
straint length is attained. 

Only a few coefficients of the generator polynomials are chosen at each step (these 
coefficients correspond to a few memory cells of the encoder). The number of codes in the search 
increases linearly with the constraint length. Basically, the complexity of the algorithm is deter- 
mined by the complexity of computing the free distance. Therefore, any class of codes whose 
free distance can be calculated can be constructed using this algorithm. 

The algorithm is composed of two parts, an initialization phase and an iterative phase. The 
initialization of the algorithm guarantees a large distance along the first and last branches of error 
events. The codes have the structure shown in Figure 2. The iterative phase constructs rate kin 
codes of constraint length v + i from only one rate kin code of constraint length v, where i < k. 
The iterative phase of the Algorithm is an exhaustive search over a very limited set of codes (2 n , 
the number of signals in the signal set). 

The initialization of the algorithm consists in constructing the ends of the codes shown in 
Figure 2. The iterative part of the algorithm consists in extending the shortest registers of the 
code. For simplicity, the algorithm is described for rate 1/n trellis codes which are long enough to 
have the structure shown in Figure 2. 

Initialization step: 

Construct the rate 1/n trellis code with constraint length V = 2q — 1 = 2n - 1 as in Fig- 
ure 2. Set j to q (the variable j indicates the position of the cell to be added to extend 
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the code). 

Iterative steps: 

(1) : Given j, to construct a rate 1 In trellis code with constraint length V + 1 from a rate 
1/n code with constraint length V insert a memory cell at position j and let it have all pos- 
sible connections to the n outputs of the binary encoder (2 n codes). 

(2) : If this memory cell is going to be the f' of the next code go to (3) (good forward dis- 
tance profile). If it is going to be the (/ + l) 11 ’ memory cell of the next code, go to (4) 
(good backward distance profile). 

(3) : Among the codes with the best free distance in the set of 2 n codes, select the one 
which has the best distance profile along the first j + 1 branches and the best distance 
profile along the last (V + 1) — j branches of error events. Increment j. Go to (1) unless 
the desired constraint length is reached. 

(4) : Among the codes with the best free distance in the set of 2 rt codes, select the one 
which has the best distance profile along the first j branches and the best distance profile 
along the last (v + 1) — j - 1 branches of error events. The value of j is unchanged. Go 
to (I) unless the desired constraint length is reached. 

Rate kin trellis codes can be constructed by embedding k of this algorithm. Table 1 
shows rate 1/2 codes. These codes are almost optimum for many constraint lengths and have free 
distances very similar to the free distances of quick-look-in codes and complementary codes even 
though the construction is much simpler. All these codes, except for very short constraint lengths 
have the structure shown in Figure 2. Table 2 gives rate 2/3 trellis coded 8-PSK schemes. These 

codes have good distance profiles from both ends of error events (although not optimum). 

* 

Recently, Porath and Aulin proposed a "fast algorithmic construction" to construct longer 
rate k/(k + 1) codes by extending B shorter codes with an optimum distance profile ( B is 
increased until the set of codes is large enough to contain a good code) [9], This algorithmic con- 
struction is not fully deterministic unless B is large (the larger B is, the more exhaustive the 
search) and searches over many more codes than our algorithm. Modifying our algorithm to 
increase the size of the set of codes examined almost always gives optimum codes. However, we 
are more interested in finding fast algorithms to construct long codes than in finding optimum 
codes in a non-deterministic way. 

Conclusion: The main criterion used to construct codes is the free distance, as opposed to other 
known algorithms which maximize the distance profile of codes. From a theoretical point of view, 
the construction technique shows that iterative construction algorithms can give codes with large 
free distance. It also shows that a compromise must be made between optimum distance profile 
and optimum free distance codes. From a practical point of view, it allows the fast construction 
of good codes for any TCM scheme. This can be the starting point for the construction of trellis 
codes optimum for specific applications. 
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BINARY ENCODER 


MAPPER 



k - k UNCODED BITS 



Table 1: Construction of rate 1/2 convolutional codes. 


V 

code 

gx 

g z 

dir 

4 - 

C 2 

QLI 3 

BJ 4 

PA 5 6 

Max 4 

1 

fzfi 

3 

2 

- 

3 

- 

- 

- 

- 

- 

2 

fzfifz 

7 

5 

2f 

5 

4 

5 

5 

5 

5 

3 

fzA A h 

17 

11 

2b 

6 

4 

6 

6 

6 

6 

4 

fzA-Afz 

33 

25 

3f 

7 

5 

7 

7 

7 

7 

5 

fzA-Afz 

63 

55 

3b 

8 

6 

8 

8 

8 

8 

6 

fzA-Afz 

143 

135 

4f 

9 

6 

9 

9 

10 

10 

7 

fzA-Afz 

313 

265 

4b 

10 

7 

9 

10 

10 

10 

8 

fzA-Afz 

613 

565 

5f 

10 

7 

10 

11 

11 

12 

9 

fzA-Afz 

1433 

1345 

5b 

12 

8 

11 

12 

12 

12 

10 

fzA-Afz 

3073 

2705 

6f 

12 

8 

12 

13 

13 

14 

11 

fzA-Afz 

6173 

5605 

7f 

14 

8 

13 

14 

14 

15 

12 

fzA-Afz 

14373 

13405 

6b 

14 

9 

14 

15 

14 

16 

13 

fzA-Afz 

30673 

27105 

7b 

16 

10 

14 

16 

16 

16 

14 

fzA-Afz 

61473 

56105 

8f 

16 

10 

15 

17 

16 

18 

15 

fzA-Afz 

143473 

134505 

8b 

18 

10 

16 

18 

- 

19 

16 

fzA—Ah 

307073 

271105 

9f 

18 

10 

16 

13 

- 

20 


1 Hamming free distance (search over 4 codes). 

2 Construction technique [10] (search over 4 codes). 

3 QLI codes [3] (search over 2 V “ 1 codes). 

4 Complementary codes [4] (search over 2 V “ 1 codes). 

5 Construction algorithm [9] (search over 32 codes). 

6 Optimum codes [11] (exhaustive search). 




Table 2: Construction of rate 2/3 trellis coded 8-PSK 
schemes. 
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ABSTRACT 


In this paper, we construct 2,4, and 8-dimensional trellis codes with rate k/k+1 
systematic binary convolutional codes and multi level / phase signal sets which have good 
distance profiles using a nested step by step search algorithm. We then simulate the Fano 
algorithm to study the effects of the distance profile on the computational load of a 
sequential decoder. The results of few such decoding simulations are presented. 

INTRODUCTION 

In some practical applications where the available signal energy is relatively high, a 
Viterbi decoder at the receiving end of the communication system would be very inefficient. 
This is because, even for high Signal to Noise Ratio(SNR), it still performs 2 M 
computations per decoded information block, where M is the encoder memory. On the 
other hand, a sequential decoder, whose computational speed is adaptable to varying noise 
levels, would decode much faster on a high SNR channel. 


This work was supported by NASA grant NAG5-557. 
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Another advantage of using a sequential decoder is that the decoding speed is 
independent of the encoder memory, whereas the number of computations performed by a 
Viterbi decoder increases exponentially as the encoder memory increases. This allows long 
constraint length codes to be used with a sequential decoder. 

The performance of a sequential decoder depends on the Column Distance 
Function(CDF) of the code being used. Also, it is the initial part of the CDF, the Distance 
Profile, which affects the performance more than the latter part. These results for binary 
convolutional codes were first discovered by Chevillat and Costello[l]. 


DEFINITIONS 


A rate k/k+1 systematic binary convolutional encoder of the form shown in 
figure[l(a)] can be described by a binary generator vector G(i), where 

G(i) = [ g ! (i) g k (i) ] T , i = 0,1, M, 

and gb(i), IS h S k, is equal to 0 if there is no connection to a mod-2 adder and equal to 1 if 
there is a connection and T stands for the transpose of the matrix. Figure[l(a)] shows the 
straight forward form of a rate2/3 systematic binary convolutional encoder and figure[ 1 (b)] 
shows an alternate form the same encoder. Here, G(0) = [0 0] T , G(l) = [1 0] T , G(2) = [0 
Of, G(3) = [1 0] T , and G(4) = [1 if. The encoder in figure[l(b)] can now be described 
by 


3 



G = 


= [ G(M) 



G(0) ] 


In figure[l(b)], = [11010] and = [10000] 

The CDF of a trellis code can be defined in the same way as for a binary 

convolutional code by replacing the Hamming weights by the Euclidean weights[2]. The 
CDF of order i, d. , is defined as 


d. = Min 
1 i * i 7 


X * a (z " ) ' a(2 'n> * 2 


n = 0 


> Mi: 


Min 


I A 

n = 0 


2 

q(£n) 


where {x,,} and {x’ n ) are two distinct information sequences, [z^ and [z' n ] = [^ + 
are the corresponding code sequences , [gj is the error sequence, (a^)} and [a(z’ n )} are 
the corresponding signal sequences, A 2 q( ^ are the Euclidean weights, and q^) is the 

number of trailing zeros in [g n ] . The distance profile of a trellis code can be defined as its 
CDF over only the first constraint length, i.e., (d Q , d v d 2 ,..,d M ). 

A trellis code is said to have a distance profile (d Q ,...d M ) superior to the distance 
profile (d' 0 ,...,d' M ) of another code of the same memory order M if for some p, 0 < p < 

M, 


d^d'j, i = 0,l,2,....,p-l 
>d'., i = p. 


Ungerboeck[3] constructed trellis codes for a Viterbi decoder. Later, Porath & Aulin 
[4] constructed mostly optimum trellis codes with either the same or a better d^. In both 

cases the goal was to optimize the free distance, although in the latter case the algorithm 
also tries to acheive a good CDF. In our search for codes, the only goal was to optimize the 
distance profile. A combining algorithm to construct codes with good distance profile is 
also described in [4]. 


THE CODE SEARCH ALGORITHM 

We employed a nested step by step search algorithm to construct codes with a good 
distance profile. Lin and Lyne[5] were the first to develop this approach which was also 
later used by Costello[6] and Hagenauer[7]. The code construction algorithm is outlined 
below. 

Step h 

Choose the initial k-bit connection vector which results in the maximum value of d Q . 
(It turns out that this vector is always G(0) = (0) for trellis codes.) 

Step 2i 

Store the information bits for the set of 2 k (one branch) paths along with then- 
distances and the initial k-bit connection vector along with the distance d Q and set i = 0, 

where i is the length of the shift registers. 

Step 3: 

Increment i and extend all the branches stored using each of the 2 k possible 
combinations of connection vectors G(i) and compute their distances dj. 
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Step 4: 

Choose the G(i) that results in the maxinrium growth of the distance profile. In case 
of a tie, the vector which resulted in the minimum number of paths with distance dj is 

chosen. 

Step 5: 

Store the information bits for all paths of length i+1 branches along with their 
distances and the connection vector G(i) along with the distance dj. If i = M, stop. 

Otherwise, repeat Step 3 until i = M, the encoder memory. 

The same algorithm can be used to construct trellis codes using multi-dimensional 
modulation schemes. In this study, trellis codes using four and eight dimensional schemes 
with an 8-PSK signal constellation were constructed. The four-dimensional schemes were 
constructed using a rate 5/6 systematic binary convolutional encoder and the eight- 
dimensional schemes use a rate 1 1/12 encoder. Due to the fact that the algorithm requires an 
enormous computational time, encoders with uncoded bits were used. In each of the two 
cases, encoders with rates 2/3 and 3/4 were used. Figure [2] shows a rate 5/6, M = 4, 
systematic binary convolutional encoder with four uncoded bits for a four-dimensional 8- 
PSK scheme. 

It should be noted that the number of paths whose distance must be computed 
increases exponentially with i. For this reason, a distance cutoff value is introduced and 
only those paths that do not exceed this cutoff value are stored for further extension. For 
short constraint length codes, a reasonably high cutoff value can be used to construct the 
codes along with their distance profiles. The cutoff value can be estimated from the distance 
profiles of the shorter constraint length codes using the following lemma. 


Lemma: 


For rate k/k+1 trellis coded modulation schemes. 


Proof: 


d i+ i - dj ^ Aq, i = 1,2,...,M-1. 


Consider the minimal form of the encoder shown in figure [1(b)]. Let dj be the 
minimum distance at the j th stage. 

Case 1 . The content of the M th stage of the shift register is a 1. Then the signal selected 
would be from the subset { S 1 ,S 3 ,...,S 2 (k+i). 1 }. which would result in an increase of A Q at 

the (j+1) th stage. 

Case 2 . The content of the M th stage of the shift register is a 0. Then, the signal selected 
would be from the subset { S 0 ,S 2 ,...,S 2 (k + i)„ 2 )> which would result in no increase at the 

(j+1) th stage. 

The algorithm employs a straight forward binary encoder to construct the codes. The 

generators given in the tables[l-5] are for the form of the encoder shown in figure[l(b)]. 
The mapping of the k+1 bit binary code symbols z n into the signal points a^) was 

done by using the set-partitioning method described in [3]. Trellis codes using rate 2/3 
(8PSK) and 3/4 (16QAM) systematic binary convolutional codes were constructed. Codes 
using four and eight dimensional modulation schemes were also constructed. 

We show in Tables[l-5] a list of 2, 4, and 8-dimensional trellis codes with rate 2/3 
and rate 3/4 systematic binary convolutional codes using 8-PSK and 16 QAM signal 
constellations. A list of codes for the rate 3/4 (2 uncoded bits), 4-dimensional scheme is not 
included here since the algorithm found codes with gw) = 0 and g^ and g^ the same as 
with 3 uncoded bits in table [3]. Figures [3 and 4] compare the distance profiles of two rate 



2/3, constraint length 9and 10 codes respectively - one was constructed with the step by 
step algorithm presented here to optimize the distance profile and the other was constructed 
by Ungerboeck[3] to optimize the free distance. Figure[5] compares the distance profiles of 
two other codes, a good distance profile code constructed here and a mostly optimal code 
constructed with the combining algorithm[8], both of which are rate 2/3, constraint length 
13 codes. 


DECODING SIMULATIONS 


To study how the distance profile affects the decoding speed or the number of 
computations required per frame by a sequential decoder, we simulated a Fano decoder and 
decoded noisy frames generated by using the rate 2/3 constraint length 9 and 10 codes and 
the rate 2/3, constraint length 13 code constructed with the step by step algorithm. We then 
used the rate 2/3 constraint length 9 and 10 codes constructed by Ungerboeck(UG) and the 
mostly optimal(MO) code of rate 2/3, and constraint length 13. 


THE SIMULATION PROCEDURE 


S.lgp.l, 

Generate a 128-bit long random sequence of l's and 0's called a frame. 

Sjg p .2, 

Encode the bits and map the binary encoder outputs into the 8-PSK signal points. 
Step 3. 

Generate random 2-dimensional Gaussian noise vectors and add them to the signal 


points. 
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Ste B-4, 

Map the received signals into 8-PSK signal points with a hard decision 
demodulator based on the decision regions shown in figure [6]. 

STEP 5. 

Decode the noisy frames with a Fano decoder. 

Each simulation involved generating a thousand random sequences of l's and 0's, 
encoding, and mapping the code words into an 8-PSK signal set. Each sequence was a 
frame of 128 symbols. A Gaussian channel was simulated to transmit these signal points. 
Hard decisions were made by the Fano decoder based on the optimum decision regions 
shown in figure[6]. Each forward look by the decoder was counted as one 
computation.The metric used was not an optimum metric, but was good enough to compare 
the computational distributions of the different codes. The bit metric used for decoding 
binary convolutional codes was used as the symbol metric for trellis codes: 

Mtr^laO^)] = log 2 [ PCrfz^la^)) / Pfr^))] - R, 

where, rf^) is a received symbol, aO^) is a transmitted symbol, and R is the code rate. 

The results of these simulations at different SNRs are shown in figures [7-12]. C is 
the number of computations required per frame. The figures show the probability that C is 
greater than or equal to a number N plotted against N. The results obtained generally agree 
with the results previously found using binary convolutional codes[l]. The code with the 
better distance profile performed fewer computations on the average. 
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CONCLUSIONS 

The results of the decoding simulations were satisfactory and generally agreed with 
the results previously found for binary convolutional codes.The codes with a better 
distance profile performed fewer computations on the average. Also, the difference 
between the performance of the codes with different distance profiles seems to be greater 
than the difference for binary convolutional codes. 

It can be concluded that when choosing a trellis code for a sequential decoder, it is 
important to choose codes with rapidly increasing distance profiles. Also, since the 

computational performance of the sequential decoder does not depend on the constraint 
length of the code, long constraint length codes which guarantee a large d^ ree can be 

used. This makes the free distance a secondary criterion when designing trellis codes for 
sequential decoding. 
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Fig[2]. A Systematic Rate 5/6, M = 4 Encoder for a 4-D TCM Scheme. 



Fig[3]. Distance profiles of two rate 2/3, constraint length 9 trellis codes. 
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Fig[4]. Distance profiles of two rate 2/3, constraint length 10 trellis codes. 



Fig[5]. Distance profiles of two rate 2/3, constraint length 13 trellis codes. 
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M 

»<’> 

g ( 2) 

«M 2 

3 

12 

0 

3.17 

4 

32 

20 

3.17 

5 

52 

0 

3.74 

6 

152 

100 

3.74 

7 

252 

0 

4.00 

8 

652 

400 

4.32 

9 

652 

1400 

4.58 

10 

652 

3400 

4.90 

11 

652 

7400 

4.90 

12 

652 

17400 

5.16 

13 

20652 

37400 

5.48 

14 

60652 

77400 

5.48 

15 

60652 

177400 

5.48 

16 

260652 

1 77400 

5.48 

17 

660652 

577400 

5.48 

18 

1660652 

577400 

5.48 

19 

3660652 

577400 

5.74 

20 

7660652 

577400 

5.74 

21 

3660652 

10577400 

6.02 

22 

23660652 

10577400 

6.02 

23 

63660652 

10577400 

6.02 

24 

163660652 

110577400 

6.33 

25 

163660652 

210577400 

6.33 


Table[1]. A list of rate 2/3 trellis codes with 8-PSK signal set.(AII generators are 

in octal.) 
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M 

o (,) 

g (2 ) 

g< 3 > 

h 2 
d M 

3 

16 

04 

04 

1.2 

4 

16 

24 

24 

1.6 

5 

56 

64 

64 

1.6 

6 

156 

164 

164 

1.6 

7 

256 

064 

264 

2.0 

8 

656 

464 

364 

2.0 

9 

1656 

0464 

1364 

2.0 

10 

3656 

2464 

1364 

2.0 

11 

7656 

6464 

5364 

2.4 

12 

17656 

16464 

15364 

2.4 

13 

37656 

16464 

15364 

2.4 

14 

37656 

56464 

15364 

2.4 

15 

137656 

056464 

015364 

2.4 


Table[2]. A list of rate 3/4 trellis codes with 16-QAM signal set.(AII generators 

are in octal.) 
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M 

g (1) 

g (2) 

d M' 

3 

12 

00 

2.0 

4 

32 

20 

2.3 

5 

72 

20 

2.3 

6 

032 

120 

2.3 

7 

232 

320 

2.9 

8 

632 

320 

2.9 

9 

1632 

0320 

2.9 

10 

0632 

2320 

3.2 

11 

4632 

4320 

3.5 

12 

14632 

04320 

3.5 

13 

24632 

04320 

3.5 

14 

64632 

04320 

3.5 

15 

164632 

004320 

3.8 

16 

364632 

204320 

4.1 

17 

764632 

204320 

4.1 

18 

1764632 

0204320 

4.1 

19 

3764632 

0204320 

4.1 

20 

5764632 

4204320 

4.7 

21 

15764632 

14204320 

4.7 

22 

35764632 

14204320 

4.7 

23 

55764632 

14204320 

4.7 


TABLE[3]. A LIST OF GOOD DISTANCE PROFILE 4-DIMENSIONAL TRELLIS 
CODES WITH AN 8-PSK SIGNAL SET AND A RATE 2/3 SYSTEMATIC BINARY 
CONVOLUTIONAL ENCODER (3 UNCODED BITS). 
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M 

g (1) 

g (2) 

d M 

3 

16 

04 

1.8 

4 

26 

04 

1.8 

5 

66 

44 

2.3 

6 

166 

044 

2.3 

7 

266 

044 

2.3 

8 

666 

044 

2.3 

9 

1266 

1044 

2.9 

10 

3266 

1044 

2.9 

11 

5266 

5044 

2.9 

12 

15266 

05044 

2.9 

13 

15266 

25044 

3.5 

14 

45266 

25044 

3.5 

15 

105266 

125044 

3.5 

16 

305266 

125044 

3.5 

17 

505266 

125044 

3.5 


TABLE[4]. A LIST OF GOOD DISTANCE PROFILE 8-DIMENSIONAL TRELLIS 
CODES WITH AN 8-PSK SIGNAL SET AND A RATE 2/3 SYSTEMATIC BINARY 
CONVOLUTIONAL ENCODERS UNCODED BITS). 

M 

g (1) 

g (2) 

g (3) 

d M 

3 

16 

04 

00 

1.8 

4 

26 

04 

00 

1.8 

5 

66 

44 

00 

2.0 

6 

166 

144 

100 

2.3 

7 

166 

344 

300 

2.3 

8 

566 

744 

300 

2.3 

9 

0566 

1744 

1300 

2.3 


TABLE[5]. A LIST OF GOOD DISTANCE PROFILE 8-DIMENSIONAL TRELLIS 
CODES WITH AN 8-PSK SIGNAL SET AND A RATE 3/4 SYSTEMATIC BINARY 
CONVOLUTIONAL ENCODERS UNCODED BITS). 
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Fig[7]. Computational Distribution for two rate 2/3, constraint length 9 trellis codes at 11 

dB. 



Fig[8]. Computational Distribution for two rate 2/3, constraint length 9 trellis codes at 8 
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Fig[9]. Computational Distribution for two rate 2/3, constraint length 10 trellis codes at 8 

dB. 



Fig[10]. Computational Distribution for two rate 2/3, constraint length 10 trellis codes at 9 

dB. 
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Fig[l 1], Computational Distribution for two rate 2/3, constraint length 13 trellis codes at 10 
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Fig[12]. Computational Distribution for two rate 2/3, constraint length 13 trellis codes at 8 
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A COMPARISON OF THE RANDOM CODING BOUNDS ON THE FREE 
EUCLIDEAN DISTANCE AND THE EVENT ERROR PROBABILITY OF 
TRELLIS CODED MODULATION SCHEMES 1 


Christian Schlegel, Marc Rouanne and Daniel J. Costello, Jr. 

Department of Electrical and Computer Engineering 
University of Notre Dame 
Notre Dame, Indiana 46556 

Abstract: A comparison of an expurgated upper bound on the event error probability of 
trellis coded modulation schemes is compared to a lower bound on the achievable mini- 
mum free Euclidean distance. It is shown that the bounds are equivalent asymptotically 
and a geometric construction method for the expurgated error exponent is given. Several 
constellations are optimized with respect to the bounds. 


1 Introduction 

In recent years bandwidth efficient Trellis Coded Mod- 
ulation (TCM) introduced by Ungerboeck [1] has be- 
come increasingly popular and much analysis has been 
devoted to the performance of these coding schemes 
on AWGN channels. It is well known that for large 
signal to noise ratios (SNR’s) the minimum free Eu- 
clidean distance df ree is the dominant parameter in 
code performance. Much research has gone into the 
search for codes with large df rte . While most of this 
work focused on finding good trellis codes with a 
given signal constellation, the constellation itself is 
also a parameter in the system design. There have 
been a few attempts to design better codes by using 
non-standard signal constellations, usually asymmet- 
ric MPSK signal sets [2]. While these codes showed 
slight performance improvements, no general rule on 
how to choose the constellation is known. 

2 Error Bounds from the Free Euclidean Dis~ 
tance 

A rate R « k/n trellis code is generated by a bi- 
nary convolutional encoder followed by a mapper as 
shown in Figure 1. The convolutional encoder is a fi- 
nite state automaton, which, at each time interval j, 
accepts k binary input bits, makes a transition from 
its state Sj at time j to one of 2 k possible successor 
states Sj+i and outputs one of 2 n binary n- tuples. 

In the minimal realization [3] the encoder consists 
of k shift registers with lengths m fc . Let the 

memory order m be min (rrij) and assume henceforth 

: This work was supported by NASA Grant NAG5-557 


that mi = m for all i . The mapper then translates 
each binary n- tuple into one of A = 2 n channel sig- 
nals from a (possibly multidimensional) signal set 
A = • • • ,&*}. The signal sets discussed in 

this paper are 4-ary pulse amplitude modulation (4- 
PAM), 8-PAM, 8-ary phase shift keyed modulation 
(8-PSK), and 16 quadrature amplitude modulation 
(16-QAM). These signal sets are shown in Figure 2. 
A rate J? = k/n trellis code transmits k bits/channel 
signal. 


k input 
bits 


n output bits 


channel 



, ( 1 ) ( 2 ) 
^“(Uj- v- 
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Figure 1: Convolutional Encoder and Mapper for a 
TCM-scheme. 


At the receiver, the signal sequences are decoded 
probabilistically using the Viterbi algorithm. The 
Viterbi algorithm is a maximum likelihood sequence 
estimator that finds the code sequence which most 
closely corresponds to the sequence of received sig- 
nals. Any decoding error in a trellis code is character- 
ized by a set of incorrect branches which the decoder 
chooses over the correct branches. With each incor- 
rect path we may associate a sequence of incorrect 
trellis states Sj, while the sequence of correct states 
is Sj. An error event of length / can be described 
by l state pairs, (5 0 , 5 0 ), . . . , (5/, Si), with 5 0 = 5 0 , 




Si as Si, and Sj ^ Sj for 0 < j < l, i.e., the in- 
correct path must not touch the correct path. As- 
sociated with these paths are two symbol sequences 

y = lUo'lLv • • • ’ &-J y - &> 4 i> • • • ’ij-J- where 

and ^ are signals from the signal set A. The prob- 
ability of an error event is a function of the cumula- 
tive Euclidean distance between the code sequences 
associated with the correct and the incorrect path. 
We may therefore write 

Pr[(5 0 , . . . , Si) -» (So, • • . . 5 ,)] = Q , 

( 1 ) 

where Q{~f) = J 7 °°l / y/2n is the complemen- 
tary error function, No is the one sided noise power 
spectral density, and ||y - y|| 2 = T!~^ 0 \gj ~ £/ is 
the norm of the difference of the two code sequences 
y and y. 


4-PAM 



8-PSK 


8-PAM 

• • • • 

• • • • 

• • • • 

• • • • 

16-QAM 


Figure 2: Constellations discussed in this paper. 

In [4] it is shown that the event error probability 
P e can be lower bounded by 

< 2 > 


where d free = min J j|y - y|| 2 /E, is the normalized 
yy v 

minimum free Euclidean distance of the code and / is 
the length of the path that achieves d/ rec . E s is the 
average signal energy E $ = Using the 

well known approximation of the Q-function [6, page 
83] 


we obtain for (2) 



+ ln ( 1 d? frtc E,/No ) ) 


= ^exp(-d} r „^2 {l+ o(E,/No))j (4) 

where o(E t /No) « \n(E 9 /No)/(E 9 /N 0 ) is a quantity 
that goes to 0 as E 9 /N Q — > oo. 

From the transfer function bound for trellis codes 
we know that the asymptotic value of the event error 
probability P e for large E 9 /Nq is approximated by 



where n/ re « is the path multiplicity of the code, i.e., 
the number of distinct error events with the minimum 
free Euclidean distance d/ ree . The free distance df ree 
thus gives us a good measure of code performance at 
high values of E 9 /Nq . 


3 Random Coding Error Bounds 

In [4], it is shown that there exists an expurgated 
upper bound on the event error probability of the 
best trellis code given by 



(2* — l)c~ iln2 \ * 
e E(.)-k In 2 _ 1 J 




where 0 < s < 1 and 


E(s) > fcln 2, 

( 6 ) 


E(s)-- In £ £ , (7) 

i sail jss 1 


Let us further define the expurgated error exponent 
E„ as 


E ex = max 


0 < s < 1 



( 8 ) 


For large memory lengths m, E zx is the exponent that 
determines code performance. 
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(3) 





ORIGINAL PAGE SS 

OF POOR QUALITY 


We thus have an upper bound (6) and a lower 
bound (4) on the event error probability. Since these 
two bounds must not contradict each other, 

^exp(-dJ rw ^(l + o(E./iVo))) < 

((2* - l) C - fctal ) * ( 9 ) 

where df ree is now the minimum free Euclidean dis- 
tance of the best trellis code. We may take the nat- 
ural logarithm of both sides and obtain 


4 Asymptotic Error Estimates 

For large m, the contribution of the term Q(E(a)) 
will become negligible in (13) and we may therefore 
ignore it. Maximizing the minimum free Euclidean 
distance in (13) is then the same as maximizing the 
expurgated error exponent E cx , given in (8), with 
the only difference being that the range of a(s) is re- 
stricted in (6). If the maximizing a max < (E $ /Nq)/ 4, 
then the resulting constellation that maximizes the 
bound on the free distance will be the same that max- 
imized the expurgated error exponent, and 
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-k ln2\ 


4 ln( y/2nl) 
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(10) 


For simplicity, let us denote sE, f No by a. Rewriting 
(10) above we obtain 


where 


/ (2* — l)e~ k]n2 \ 

^ lll2 Yj 


Note that 
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(16) 


(17) 
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( 11 ) 


where 


0 (E(a)) = in ( e £ (“>-* toJ -l) - In (l - e'* 1 ” 2 ) 


E(a) = -lnX:X:3(fl < )?(fl>)e- a|a *- 4 ' |J , (12) 
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0 < a < 


Es/No 
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In order to obtain a lower bound on the minimum 
free Euclidean distance df ree , we let E,/No — ► oo in 
(11) and obtain the same bound as Rouanne [5], i.e., 


d)ree > 


max 
0 < a 


mE(a) + Q(E(a)) \ 
a a ) 


; E(a) > k In 2. 


(13) 


If a max = ( E s /No)/4 , then the maximizing parame- 
ter in (8) will be s = 1. The error bound (6) then 
becomes 

(*>k _ <i\ e -k\n2 

P'Z Ito-tL.-L e ~ m Ro > k\n2, (18) 

where 

Ro = - In ££g(aj 9 (a J )e _i! ^ E * /iVo (19) 

.= l;=l 

is the cutoff-rate of the constellation in nats/signal. 
Computer evaluations of (8), or equivalently of (13) 
with m — ► oo, show that E(s) = E(a) —> Hn2 for 
a — ► a max . This observation holds true for all con- 
stellations we have analyzed. 

Figure 3 shows the expurgated error exponent E cx 
for 4-PAM as a function of EJNq. Since a max is 
independent of E a /N 0 (17) shows that E ex is a lin- 
ear function of E 9 /N 0 . The slope of this function, 
E(ot max )/( 4a max ), is a function of k because a max de- 
pends on k. As E s /N 0 approaches 4a max from above, 
E ex — ► Ro. 4 a m ax is the smallest E 9 /Nq for which 


there exists an upper bound of the form (6). The ex- 
ponent of the upper bound at this point is Rq. The 
higher the available energy, i.e., the larger E t /N 0 for 
a particular k, the larger the expurgated error ex- 
ponent will be. For a code with a larger value of 
k, the expurgated exponent will grow more slowly 
with E 9 /Nq and a larger minimum E 9 /Nq is required 
to guarantee the exponential bound. Rq retains its 
importance as a cutoff-rate, since it is not only the 
expurgated exponent at the cutoff-point E s /N 0 = 
4a ma x» but also the maximum rate k = Rq/ In 2 for 
which (6) exists. Thus the expurgated error exponent 
as well as the asymptotic part of the free distance 
bound (13) can be easily constructed from a graph of 
the cutoff-rate Rq in the following manner. The con- 
struction is illustrated in Figure 3 for rate k = 1.99 
bits/symbol for 4- PAM signalling. 


^ [oats/signal] 



Figure 3: Cutoff-rate Rq and expurgated error expo- 
nent of 4-PAM. 


(1) Choose the value of k (which determines the 
rate either in nats/signal or bits/signal). The 
cutoff-point is the intersection of a line parallel 
to the J?,/jVo-axis at a distance k (in nats/signal) 
with the cutoff-rate curve. The x- value of the 
cutoff-point is 4 a mttX . 

(2) Connect the origin of the graph with the cutoff- 

point in a line g . 

(3) The expurgated exponent for any E s /Nq > 4 a max 

is the y-value of g at that £ a /iVo- value. 

(4) The asymptotic bound on d* Tee from equation 
(14) is 4m x the slope of g. 


5 Constellation Optimization 

An important parameter in (13) is the constellation 
itself, that is, the coordinates of the signal points in 
the set A . We have already observed that for m — ► oo 
and compatible values of fc, the maximization of the 
minimum free Euclidean distance and the expurgated 
error exponent will yield the same signal constella- 
tion. The cutoff-rate Rq of rectangular constellations 
(16-QAM etc.) is higher than Rq of their constant en- 
velope counterparts (16-PSK, etc.) in the moderate 
E 9 /Nq- range. It should be obvious from the expo- 
nent construction that constellations with a better 
cutoff-rate at a given E 9 /Nq will also yield a better 
expurgated error exponent as well as a better bound 
on df ret . 

Next we have optimized the constellations intro- 
duced in Figure 2 using a program implementing the 
Newton algorithm to find the global minimum of a 
function of n variables. Figure 4 shows the results 
for the 4-PAM constellation and the 8-PAM constel- 
lation for a number of rates. It is interesting to note 
that for small rates k both constellations converge 
towards binary signalling (BPSK). 

4-PAM —1— — 

optimized 4-PAM 
k : = 0.2 
0.6 
0.8 
1 
2 

8-PAM 

optimized 8-PAM 
k:= 0.2 
0.6 
0.8 
1 
2 

Figure 4: Optimized 4-PAM and 8-PAM constel- 
lations for different rates k. 

Figure 5 shows the optimization for 8-PSK. Con- 
trary to the PAM signed sets, the maximizing con- 
stellation does not vary with E t /N 0 . The angles are 
slightly shifted towards asymmetric 8-PSK. 

Figure 6 shows an optimized 16-QAM for rate 
k = 3. As the two dimensions have been maximized 
under the same constraints, the constellation is the 
cartesian product of two 4-PAM constellations. 
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Figure 5: Optimized 8-PSK signal set. 
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Figure 6: Example of an optimized 2-dimensional sig- 
nal set for rate k = 3. 
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